Re: [galaxy-dev] Regarding galaxy wrappers: lumi, DESeq edgeR
(Sachit, please don't cross-post to the dev and user lists?) There's a beta version of an edgeR/deseq/voom wrapper and a companion htseq based count matrix maker in the test toolshed statistics section owned by me - fubar. Definitely not quite ready for production but installable and I'd appreciate feedback from anyone brave enough to try them out. Best installed on a test instance you plan on destroying because the R 3.0 dependency is about to swap over to Bjorns better one - should be done later this week when I get some time. Be warned it takes a long time to auto-install - 10-15 minutes or so on my laptop. On Mon, Jul 8, 2013 at 3:36 PM, Sachit Adhikari sachit.techner...@gmail.com wrote: Hi, These are the R libraries. Have anyone written a wrapper for these libraries lumi, DESeq edgeR? Thanks, Sachit ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/ -- Ross Lazarus MBBS MPH; Head, Medical Bioinformatics, BakerIDI; Tel: +61 385321444 http://scholar.google.com/citations?hl=enuser=UCUuEM4J ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
[galaxy-dev] Galaxy Server Processes Dying?
Hi, Galaxy Developers, I have what I'm hoping is a fairly simple inquiry for the Galaxy community; basically, our production Galaxy server processes appear to be dying off over time. Our production Galaxy instance implements apache web scaling features so I have a number of server processes, for example my apache Apache configuration has: BalancerMember http://127.0.0.1:8080 BalancerMember http://127.0.0.1:8081 BalancerMember http://127.0.0.1:8082 BalancerMember http://127.0.0.1:8083 BalancerMember http://127.0.0.1:8084 BalancerMember http://127.0.0.1:8085 Nothing unconventional as I understand it. Similarly, my galaxy config has matching [server:ws3], [server:ws2] configuration blocks for each of these processes. When I restart Galaxy, everything is all fine and good. I'll see a server listening on each one of these ports (if I do something like lsof -i TCP -P, for example). What appears to be happening, is that for whatever reason, these server processes seem to die off over time (i.e eventually nothing is listening on ports 8080-8085). This process can take days, and at the time when no servers are available, Apache will begin throwing 503 service unavailable errors. I am fairly confident this process is gradual, for example I just checked now and the Galaxy was still available, however one server had died (the one on TCP port 8082). I do do have a single separate job manager and two job handlers; at this point I believe this problem to be related to the servers only (i.e. the job manager and job handlers do not app! ear to be crashing). Now, I believe that late last week I might have 'caught' the last server process dying, just by coincidence, although I am not 100% certain. Here is the Traceback as it occurred: galaxy.jobs.runners.pbs DEBUG 2013-07-02 08:47:12,011 (6822/39485.sc01) PBS job state changed from Q to R galaxy.jobs.runners.pbs DEBUG 2013-07-02 08:54:36,565 (6822/39485.sc01) PBS job state changed from R to C galaxy.jobs.runners.pbs DEBUG 2013-07-02 08:54:36,566 (6822/39485.sc01) PBS job has completed successfully galaxy.jobs DEBUG 2013-07-02 08:54:36,685 Tool did not define exit code or stdio handling; checking stderr for success galaxy.datatypes.metadata DEBUG 2013-07-02 08:54:36,812 loading metadata from file for: HistoryDatasetAssociation 6046 galaxy.jobs DEBUG 2013-07-02 08:54:38,153 job 6822 ended galaxy.jobs.runners.pbs DEBUG 2013-07-02 08:54:49,130 (6812/39473.sc01) PBS job state changed from R to E galaxy.jobs.runners.pbs DEBUG 2013-07-02 08:54:52,267 (6812/39473.sc01) PBS job state changed from E to C galaxy.jobs.runners.pbs ERROR 2013-07-02 08:54:52,267 (6812/39473.sc01) PBS job failed: Unknown error: -11 galaxy.jobs.runners ERROR 2013-07-02 08:54:52,267 (unknown) Unhandled exception calling fail_job Traceback (most recent call last): File /group/galaxy/galaxy-dist/lib/galaxy/jobs/runners/__init__.py, line 58, in run_next method(arg) File /group/galaxy/galaxy-dist/lib/galaxy/jobs/runners/pbs.py, line 560, in fail_job if pbs_job_state.stop_job: AttributeError: 'AsynchronousJobState' object has no attribute 'stop_job' Now, I have some questions regarding this issue; 1) It appears to me that although this is a sub-optimal solution, restarting Galaxy solves this problem (i.e. server processes will be listening after restarting Galaxy). Is it possible, or safe, or sane to just restart a single server on a singe port? Ideally I would actually like to fix the problem that is causing my server processes to crash, although I figured it wouldn't hurt to ask this question regardless. 2) Similar to the question above, is it possible to configure Galaxy in a way that server processes re-spawn in a self-service manner (i.e. is this a feature of Galaxy, for example, because server processes dying regularly is either a known issue or expected and tolerable (but undesired) behaivor)? 3) To me, the error messages above aren't very meaningful, other than the Traceback appears to be PBS-related. Would anybody be able comment on the problem above (i.e. have you seen something like this), or comment on Galaxy server processes dying in general? I have done some brief searching of the Galaxy mailing list for server crashes and did not find anything suggesting this is a common problem. 4) I am not 100% confident at this point that the Traceback above is what killed the server process. Does anybody know of a specific string I can search for (a literal) to identify when a server process actually dies? I have done some basic review of log data (our Galaxy server generates lots of logs), and Traceback does not appear to be a valid string to uniquely identify a server crash (they occur too frequently). I currently have logging configured at DEBUG. In case this is relevant, I am using the following change set for Galaxy: hg parents changeset: 9320:47ddf167c9f1 branch: stable tag: tip user:Nate Coraor
Re: [galaxy-dev] Regarding galaxy wrappers: lumi, DESeq edgeR
Sorry about that Ross. I would try your wrapper of edgeR and would happily report you the bug. Can you make me the link of Galaxy Test Toolshed? Thanks, Sachit On Mon, Jul 8, 2013 at 11:51 AM, Ross ross.laza...@gmail.com wrote: (Sachit, please don't cross-post to the dev and user lists?) There's a beta version of an edgeR/deseq/voom wrapper and a companion htseq based count matrix maker in the test toolshed statistics section owned by me - fubar. Definitely not quite ready for production but installable and I'd appreciate feedback from anyone brave enough to try them out. Best installed on a test instance you plan on destroying because the R 3.0 dependency is about to swap over to Bjorns better one - should be done later this week when I get some time. Be warned it takes a long time to auto-install - 10-15 minutes or so on my laptop. On Mon, Jul 8, 2013 at 3:36 PM, Sachit Adhikari sachit.techner...@gmail.com wrote: Hi, These are the R libraries. Have anyone written a wrapper for these libraries lumi, DESeq edgeR? Thanks, Sachit ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/ -- Ross Lazarus MBBS MPH; Head, Medical Bioinformatics, BakerIDI; Tel: +61 385321444 http://scholar.google.com/citations?hl=enuser=UCUuEM4J ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
Re: [galaxy-dev] Regarding galaxy wrappers: lumi, DESeq edgeR
Hi Sachit, It's packaged up so it installs from the test toolshed directly via the admin interface of your test instance, but you'll need a good reliable network as there is a lot that gets downloaded and installed. Like other toolshed tools, it would be possible but not advisable to download and unpack the archive at http://testtoolshed.g2.bx.psu.edu/view/fubar/edger_test and install manually but your instance will need R 3.0.1 with bioconductor/edgeR/deseq/limma installed - if it's your first try, it will probably take some time and effort to read all the relevant docs to do that or the R/BioC mailing lists can provide assistance if you need it. The edgeR tool requires a count matrix (reads per sample per contig) as input which the htseq tool http://testtoolshed.g2.bx.psu.edu/view/fubar/htseq_bams_to_count_matrix will generate from any number of aligned bam/sam files and a GTF file with the appropriate gene model. Again, automatic installation from the toolshed is highly recommended, otherwise you'll need to manually install the tool and the dependencies. Note that edgeR requires replicate samples in each comparison group so the absolute minimum is 4 bam files as input to the count matrix maker. p values from analyses without replicates are not interpretable in any biologically meaningful way and are meaningless in terms of the usual frequentist interpretation in my opinion. On Mon, Jul 8, 2013 at 4:28 PM, Sachit Adhikari sachit.techner...@gmail.com wrote: Sorry about that Ross. I would try your wrapper of edgeR and would happily report you the bug. Can you make me the link of Galaxy Test Toolshed? Thanks, Sachit On Mon, Jul 8, 2013 at 11:51 AM, Ross ross.laza...@gmail.com wrote: (Sachit, please don't cross-post to the dev and user lists?) There's a beta version of an edgeR/deseq/voom wrapper and a companion htseq based count matrix maker in the test toolshed statistics section owned by me - fubar. Definitely not quite ready for production but installable and I'd appreciate feedback from anyone brave enough to try them out. Best installed on a test instance you plan on destroying because the R 3.0 dependency is about to swap over to Bjorns better one - should be done later this week when I get some time. Be warned it takes a long time to auto-install - 10-15 minutes or so on my laptop. On Mon, Jul 8, 2013 at 3:36 PM, Sachit Adhikari sachit.techner...@gmail.com wrote: Hi, These are the R libraries. Have anyone written a wrapper for these libraries lumi, DESeq edgeR? Thanks, Sachit ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/ -- Ross Lazarus MBBS MPH; Head, Medical Bioinformatics, BakerIDI; Tel: +61 385321444 http://scholar.google.com/citations?hl=enuser=UCUuEM4J -- Ross Lazarus MBBS MPH; Head, Medical Bioinformatics, BakerIDI; Tel: +61 385321444 http://scholar.google.com/citations?hl=enuser=UCUuEM4J ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
Re: [galaxy-dev] [galaxy-user] Regarding galaxy wrappers: lumi, DESeq edgeR
Hi Sachit, we are working currently on a DESeq2 wrapper. Not yet finised sorry. There is also a edgeR Test repository from Ross Lazarus under: http://testtoolshed.g2.bx.psu.edu/view/fubar/edger_test Work is in progress, help is welcome! Ciao, Bjoenr Hi, These are the R libraries. Have anyone written a wrapper for these libraries lumi, DESeq edgeR? Thanks, Sachit ___ The Galaxy User list should be used for the discussion of Galaxy analysis and other features on the public server at usegalaxy.org. Please keep all replies on the list by using reply all in your mail client. For discussion of local Galaxy instances and the Galaxy source code, please use the Galaxy Development list: http://lists.bx.psu.edu/listinfo/galaxy-dev To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/ ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
Re: [galaxy-dev] [galaxy-user] Regarding galaxy wrappers: lumi, DESeq edgeR
Thank you Björn and Ross. I will install the tools from the admin interface. On Mon, Jul 8, 2013 at 12:35 PM, Björn Grüning bjoern.gruen...@pharmazie.uni-freiburg.de wrote: Hi Sachit, we are working currently on a DESeq2 wrapper. Not yet finised sorry. There is also a edgeR Test repository from Ross Lazarus under: http://testtoolshed.g2.bx.psu.edu/view/fubar/edger_test Work is in progress, help is welcome! Ciao, Bjoenr Hi, These are the R libraries. Have anyone written a wrapper for these libraries lumi, DESeq edgeR? Thanks, Sachit ___ The Galaxy User list should be used for the discussion of Galaxy analysis and other features on the public server at usegalaxy.org. Please keep all replies on the list by using reply all in your mail client. For discussion of local Galaxy instances and the Galaxy source code, please use the Galaxy Development list: http://lists.bx.psu.edu/listinfo/galaxy-dev To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/ ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
Re: [galaxy-dev] Galaxy Server Processes Dying?
Hi Dan, That's old code. Updating will probably help. Logging level just takes disk space, but just in case you haven't followed http://wiki.galaxyproject.org/Admin/Config/Performance/ProductionServer?action=showredirect=Admin%2FConfig%2FPerformance leaving debug = True uncommented used to fill all available server process RAM eventually AFAIK. If not already done, try # Debug enables access to various config options useful for development and # debugging: use_lint, use_profile, use_printdebug and use_interactive. It # also causes the files used by PBS/SGE (submission script, output, and error) # to remain on disk after the job is complete. Debug mode is disabled if # commented, but is uncommented by default in the sample config. # debug = True Hope this helps? On Mon, Jul 8, 2013 at 4:26 PM, Dan Sullivan dansulli...@gmail.com wrote: Hi, Galaxy Developers, I have what I'm hoping is a fairly simple inquiry for the Galaxy community; basically, our production Galaxy server processes appear to be dying off over time. Our production Galaxy instance implements apache web scaling features so I have a number of server processes, for example my apache Apache configuration has: BalancerMember http://127.0.0.1:8080 BalancerMember http://127.0.0.1:8081 BalancerMember http://127.0.0.1:8082 BalancerMember http://127.0.0.1:8083 BalancerMember http://127.0.0.1:8084 BalancerMember http://127.0.0.1:8085 Nothing unconventional as I understand it. Similarly, my galaxy config has matching [server:ws3], [server:ws2] configuration blocks for each of these processes. When I restart Galaxy, everything is all fine and good. I'll see a server listening on each one of these ports (if I do something like lsof -i TCP -P, for example). What appears to be happening, is that for whatever reason, these server processes seem to die off over time (i.e eventually nothing is listening on ports 8080-8085). This process can take days, and at the time when no servers are available, Apache will begin throwing 503 service unavailable errors. I am fairly confident this process is gradual, for example I just checked now and the Galaxy was still available, however one server had died (the one on TCP port 8082). I do do have a single separate job manager and two job handlers; at this point I believe this problem to be related to the servers only (i.e. the job manager and job handlers do not app! ear to be crashing). Now, I believe that late last week I might have 'caught' the last server process dying, just by coincidence, although I am not 100% certain. Here is the Traceback as it occurred: galaxy.jobs.runners.pbs DEBUG 2013-07-02 08:47:12,011 (6822/39485.sc01) PBS job state changed from Q to R galaxy.jobs.runners.pbs DEBUG 2013-07-02 08:54:36,565 (6822/39485.sc01) PBS job state changed from R to C galaxy.jobs.runners.pbs DEBUG 2013-07-02 08:54:36,566 (6822/39485.sc01) PBS job has completed successfully galaxy.jobs DEBUG 2013-07-02 08:54:36,685 Tool did not define exit code or stdio handling; checking stderr for success galaxy.datatypes.metadata DEBUG 2013-07-02 08:54:36,812 loading metadata from file for: HistoryDatasetAssociation 6046 galaxy.jobs DEBUG 2013-07-02 08:54:38,153 job 6822 ended galaxy.jobs.runners.pbs DEBUG 2013-07-02 08:54:49,130 (6812/39473.sc01) PBS job state changed from R to E galaxy.jobs.runners.pbs DEBUG 2013-07-02 08:54:52,267 (6812/39473.sc01) PBS job state changed from E to C galaxy.jobs.runners.pbs ERROR 2013-07-02 08:54:52,267 (6812/39473.sc01) PBS job failed: Unknown error: -11 galaxy.jobs.runners ERROR 2013-07-02 08:54:52,267 (unknown) Unhandled exception calling fail_job Traceback (most recent call last): File /group/galaxy/galaxy-dist/lib/galaxy/jobs/runners/__init__.py, line 58, in run_next method(arg) File /group/galaxy/galaxy-dist/lib/galaxy/jobs/runners/pbs.py, line 560, in fail_job if pbs_job_state.stop_job: AttributeError: 'AsynchronousJobState' object has no attribute 'stop_job' Now, I have some questions regarding this issue; 1) It appears to me that although this is a sub-optimal solution, restarting Galaxy solves this problem (i.e. server processes will be listening after restarting Galaxy). Is it possible, or safe, or sane to just restart a single server on a singe port? Ideally I would actually like to fix the problem that is causing my server processes to crash, although I figured it wouldn't hurt to ask this question regardless. 2) Similar to the question above, is it possible to configure Galaxy in a way that server processes re-spawn in a self-service manner (i.e. is this a feature of Galaxy, for example, because server processes dying regularly is either a known issue or expected and tolerable (but undesired) behaivor)? 3) To me, the error messages above aren't very meaningful, other than the Traceback appears to be PBS-related. Would anybody be able comment on the problem above (i.e. have you
Re: [galaxy-dev] gzipped fastq reader
On Thu, Jul 4, 2013 at 9:49 PM, Robert Baertsch robert.baert...@gmail.com wrote: Dan, Do these readers support gzip files? reader = fastqVerboseErrorReader reader = fastqReader Presumably you are writing a Python script using this library? The answer is a qualified yes. Instead of passing them a normal file handle using open(example.fastq) you instead use gzip.open(example.fastq) via import gzip. Do I have to define a special type in galaxy for gzipped files or will the fastq type be ok? This needs a special file format - but you are not the first person to look at this, some groups have defined custom gzipped variants of the FASTQ formats within their own Galaxy instances. I've not done this but there should be some useful emails in the archive. Note you'd also need to modify any tool definitions to that they can accept a gzipped FASTQ file. Ideally, I would like to keep my files zipped and not have galaxy unzip them, since they triple in size when unzipped. I'm happy to do a push request if you don't support this but I want to make sure I'm in line with your roadmap. Personally I would like a more general system in Galaxy for potentially any file type to be held compressed in a range of formats (e.g. using gzip, bgzf, xy, bz2, etc), with exclusions for things like BAM which are already compressed. This way naive tools would get the gzipped file file uncompressed to a temporary folder before use (i.e. no change for the tool wrapper), but if a tool accepts a gzipped file it will get that (less disk IO and CPU usage, but requires updating tool wrappers). That idea is quite ambitious through ;) I have written a simple tool to convert Illumina fastq to mapsplice fastq. Does that already exist already somewhere? I don't know. Peter ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
Re: [galaxy-dev] [GSoC2013] Week 2 Accomplishments and Week 3 Plans
Week 3 and Week 4 Plans: http://galaxy-gsoc2013.blogspot.in/2013/07/week-3-accomplishments-and-week-4-plans.html On 6 July 2013 16:46, Saket Choudhary sake...@gmail.com wrote: Here is a short post explaining the data model. Not ver elaborately written, so incase you you find a mistake or something poorly explained , please drop a comment. http://galaxy-gsoc2013.blogspot.in/2013/07/understanding-datamodel-of-galaxy.html On 1 July 2013 16:06, Saket Choudhary sake...@gmail.com wrote: Not much accomplished this week. I plan to write one more blogspot detailing the datamodel I learnt about galaxy. http://galaxy-gsoc2013.blogspot.com/2013/07/week-2-accomplishments-and-week-3-plans.html On 24 June 2013 14:11, Saket Choudhary sake...@gmail.com wrote: Hi Everyone, Just wrote a blog post for my Week1 tasks and Week 2 to-dos: http://galaxy-gsoc2013.blogspot.in/2013/06/week-1-accomplishments-and-week-2-plans.html I am maintaining a work log in this doc : https://docs.google.com/document/d/1VCKCpKW6C_Tt1DYWi43GAN_Jo-nebkkj8Ctq7WLIZFM/edit If you have any suggestions for the VCF4.1 issue or otherwise. please do let me know. Thanks Saket Choudhary Undergraduate Dept. of Chemical Engineering IIT Bombay ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
Re: [galaxy-dev] CloudMan Error
Hi guys, Any thoughts on this? I'm kind of stuck. (Even some pointers on where to look for more clues would be extremely helpful.) Thanks, Greg On Fri, Jul 5, 2013 at 11:10 AM, greg margeem...@gmail.com wrote: Hi guys, I'm hitting an error using CloudMan using the Share-an-Instance option. It says: Error creating volume from shared cluster's snapshot '['snap-cfa775ba']': 'filesystems'. Also disk stats says 0 /0 and the Applications light is yellow while the data light is green. I'm using the share string cm-808d863548acae7c2328c39a90f52e29/shared/2012-09-17--19-47 It's always worked in the past. Thanks, Greg Here's the full log: 14:58:18 - Master starting 14:58:20 - Completed the initial cluster startup process. This is a new cluster; waiting to configure the type. 14:58:24 - Migration service prerequisites OK; starting the service 14:58:24 - SGE service prerequisites OK; starting the service 14:58:31 - Setting up SGE... 14:58:51 - HTCondor service prerequisites OK; starting the service 14:58:51 - HTCondor config file /etc/condor/condor_config not found! 14:58:59 - Hadoop service prerequisites OK; starting the service 14:59:48 - Done adding Hadoop service; service running. 15:01:45 - Error creating volume from shared cluster's snapshot '['snap-cfa775ba']': 'filesystems' ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
Re: [galaxy-dev] RFC: Cheetah template in tool_dependencies.xml
It probably makes sense to specify the template type in this case.
Maybe an additional attribute, language='cheetah' ?
On Sun, Jul 7, 2013 at 1:19 PM, Björn Grüning
bjoern.gruen...@pharmazie.uni-freiburg.de wrote:
action type=template_command
#if env.get('X11_LIB_DIR', False) and env.get('X11_INCLUDE_DIR', False):
./configure --prefix=$env.INSTALL_DIR
--x-includes=$env.X11_INCLUDE_DIR --x-libraries=$env.X11_LIB_DIR
#else:
./configure --prefix=$env.INSTALL_DIR
#end if
/action
--
James Taylor, Assistant Professor, Biology/CS, Emory University
___
Please keep all replies on the list by using reply all
in your mail client. To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
http://lists.bx.psu.edu/
To search Galaxy mailing lists use the unified search at:
http://galaxyproject.org/search/mailinglists/
Re: [galaxy-dev] RFC: Cheetah template in tool_dependencies.xml
Hi James,
It probably makes sense to specify the template type in this case.
Maybe an additional attribute, language='cheetah' ?
My first suggestion was action type=cheetah_command to reflect the
template engine. I'm fine with both.
On Sun, Jul 7, 2013 at 1:19 PM, Björn Grüning
bjoern.gruen...@pharmazie.uni-freiburg.de wrote:
action type=template_command
#if env.get('X11_LIB_DIR', False) and env.get('X11_INCLUDE_DIR', False):
./configure --prefix=$env.INSTALL_DIR
--x-includes=$env.X11_INCLUDE_DIR --x-libraries=$env.X11_LIB_DIR
#else:
./configure --prefix=$env.INSTALL_DIR
#end if
/action
--
James Taylor, Assistant Professor, Biology/CS, Emory University
___
Please keep all replies on the list by using reply all
in your mail client. To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
http://lists.bx.psu.edu/
To search Galaxy mailing lists use the unified search at:
http://galaxyproject.org/search/mailinglists/
Re: [galaxy-dev] How were liftover and nib files generated?
liftOver files come from the UCSC genome browser, derived from pairwise alignments. A variety of tools can create nib files from fasta, however we prefer the use of the twoBit format now, which can be created using faToTwoBit from the UCSC utilities. -- James Taylor, Assistant Professor, Biology/CS, Emory University On Sat, Jul 6, 2013 at 9:19 PM, Perez, Ricardo ricky_...@neo.tamu.edu wrote: Dear all, When I rsync files from let say the following command: rsync -avzP rsync://datacache.g2.bx.psu.edu/indexes/dm3 . The system will sync files that are related to genome aligments, liftover, and nibs. How did the Galaxy team generated this files? The reason I ask is because I would lather will like to generate similar files to custom genomes to be used in galaxy. Thank you for your time, --Ricardo Perez ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/ ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
Re: [galaxy-dev] RFC: Cheetah template in tool_dependencies.xml
I'll soon be committing a slightly altered version of your patch which includes
a language attribute for the template_command action type.
On Jul 8, 2013, at 10:02 AM, Björn Grüning
bjoern.gruen...@pharmazie.uni-freiburg.de wrote:
Hi James,
It probably makes sense to specify the template type in this case.
Maybe an additional attribute, language='cheetah' ?
My first suggestion was action type=cheetah_command to reflect the
template engine. I'm fine with both.
On Sun, Jul 7, 2013 at 1:19 PM, Björn Grüning
bjoern.gruen...@pharmazie.uni-freiburg.de wrote:
action type=template_command
#if env.get('X11_LIB_DIR', False) and env.get('X11_INCLUDE_DIR', False):
./configure --prefix=$env.INSTALL_DIR
--x-includes=$env.X11_INCLUDE_DIR --x-libraries=$env.X11_LIB_DIR
#else:
./configure --prefix=$env.INSTALL_DIR
#end if
/action
--
James Taylor, Assistant Professor, Biology/CS, Emory University
___
Please keep all replies on the list by using reply all
in your mail client. To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
http://lists.bx.psu.edu/
To search Galaxy mailing lists use the unified search at:
http://galaxyproject.org/search/mailinglists/
___
Please keep all replies on the list by using reply all
in your mail client. To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
http://lists.bx.psu.edu/
To search Galaxy mailing lists use the unified search at:
http://galaxyproject.org/search/mailinglists/
[galaxy-dev] Error Uploading Directory of Files
Hi, Our lab recently installed a local version of Galaxy on a mid-2012 Mac Pro computer. We can access the Galaxy server and sign in as an administrator. Today we tried creating a Data Library, adding a dataset to it, and uploading a directory of files. We followed this Galaxy documentation ( http://wiki.galaxyproject.org/Admin/DataLibraries/UploadingLibraryFiles?action=showredirect=Admin%2FData+Libraries%2FUploading+Library+Files ) to setup this feature: - Admin Data Library Add datasets Upload directory of files - file format was set to auto-detect - and we chose the option to link to files instead of copying them Galaxy confirmed that the files were successfully uploaded. However, in the data library, under the Message column, is a message in red saying Job error (click name for more info). Clicking on one of the uploaded files displays a page with this information: Date uploaded: 2013-06-28 File size: 7.5 GB Data type: auto Build: sacCer2 Miscellaneous information: Traceback (most recent call last): File /Users/administrator/galaxy-dist/tools/data_source/upload.py, line 386, in __main__() File /Users/administrator/galaxy-dist/tools/data_source/upload.py, line 357, in __main__ output_paths = Job Standard Error Traceback (most recent call last): File /Users/administrator/galaxy-dist/tools/data_source/upload.py, line 386, in __main__() File /Users/administrator/galaxy-dist/tools/data_source/upload.py, line 357, in __main__ output_paths = parse_outputs( sys.argv[4:] ) File /Users/administrator/galaxy-dist/tools/data_source/upload.py, line 64, in parse_outputs id, files_path, path = arg.split( ':', 2 ) ValueError: need more than 1 value to unpack error Database/Build: sacCer2 Number of data lines: None Disk file: /Volumes/G-SPEED Q/data/Person 2012 project/DCP2 mef.fastq Questions: 1. Should we be concerned about this error? 2. If so, what is the right way to fix it? 3. If not, how do we remove the red error message next to each file: Job error (click name for more info) ? Thank you ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
Re: [galaxy-dev] Error Uploading Directory of Files
On Mon, Jul 8, 2013 at 3:36 PM, Nicholas Kline nx...@case.edu wrote: Hi, Our lab recently installed a local version of Galaxy on a mid-2012 Mac Pro computer. We can access the Galaxy server and sign in as an administrator. Today we tried creating a Data Library, adding a dataset to it, and uploading a directory of files. We followed this Galaxy documentation ( http://wiki.galaxyproject.org/Admin/DataLibraries/UploadingLibraryFiles?action=showredirect=Admin%2FData+Libraries%2FUploading+Library+Files ) to setup this feature: - Admin Data Library Add datasets Upload directory of files - file format was set to auto-detect - and we chose the option to link to files instead of copying them Galaxy confirmed that the files were successfully uploaded. However, in the data library, under the Message column, is a message in red saying Job error (click name for more info). Clicking on one of the uploaded files displays a page with this information: Date uploaded: 2013-06-28 File size: 7.5 GB Data type: auto Build: sacCer2 Miscellaneous information: Traceback (most recent call last): File /Users/administrator/galaxy-dist/tools/data_source/upload.py, line 386, in __main__() File /Users/administrator/galaxy-dist/tools/data_source/upload.py, line 357, in __main__ output_paths = Job Standard Error Traceback (most recent call last): File /Users/administrator/galaxy-dist/tools/data_source/upload.py, line 386, in __main__() File /Users/administrator/galaxy-dist/tools/data_source/upload.py, line 357, in __main__ output_paths = parse_outputs( sys.argv[4:] ) File /Users/administrator/galaxy-dist/tools/data_source/upload.py, line 64, in parse_outputs id, files_path, path = arg.split( ':', 2 ) ValueError: need more than 1 value to unpack error Database/Build: sacCer2 Number of data lines: None Disk file: /Volumes/G-SPEED Q/data/Person 2012 project/DCP2 mef.fastq I have a hunch that the problem could be related to the spaces in the input filename. In this case perhaps the upload.py script is being called without the filename being quoted or escaped. Could you retry from a nice space free import directory? Also does the path where you have installed Galaxy have spaces? Even if Galaxy itself copes, it is highly likely that one or more underlying tools wrapped for use within Galaxy would fail. [I've had minor problems with this with various tools on the Mac outside of Galaxy usage] Regards, Peter ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
[galaxy-dev] ToolShed dependencies KeyError: 'invalid_repository_dependencies'
Hi Greg, Previously my workflows had hard coded dependencies on the main Tool Shed, which currently is a bit of a problem on the Test Tool Shed. Therefore I'm update my Tool Shed workflows using a repository_dependencies.xml with implicit repository and revision fields (i.e. the latest revision within the current ToolShed). It worked for one example but not the other - this turned out to be my own error, but the Tool Shed did not handle it gracefully. Perhaps this class of error could be caught instead? Thanks! This worked, secreted_protein_workflow, Revision: 2:aecb871dfe4c http://testtoolshed.g2.bx.psu.edu/view/peterjc/secreted_protein_workflow Input: ?xml version=1.0? repositories description=This requires my SignalP and TMHMM wrapers, and my FASTA filtering tool. !-- Revision 15:6abd809cefdd on the main tool shed is v0.2.4, the current latest - but older should be OK -- repository name=tmhmm_and_signalp owner=peterjc / !-- Revision 2:abdd608c869b on the main tool shed is v0.0.5, the current latest - but older should be OK -- repository name=seq_filter_by_id owner=peterjc / /repositories This failed, rxlr_venn_workflow, Revision: 3:65d174589251 http://testtoolshed.g2.bx.psu.edu/view/peterjc/rxlr_venn_workflow On uploading the tar-ball I got this error in the browser: quote Internal Server Error Galaxy was unable to sucessfully complete your request URL: http://testtoolshed.g2.bx.psu.edu/upload/upload?repository_id=20a9f7389ced5696 Module galaxy.web.framework.middleware.error:149 in __call__ app_iter = self.application(environ, sr_checker) Module paste.debug.prints:106 in __call__ environ, self.app) Module paste.wsgilib:543 in intercept_output app_iter = application(environ, replacement_start_response) Module paste.recursive:84 in __call__ return self.application(environ, start_response) Module paste.httpexceptions:633 in __call__ return self.application(environ, start_response) Module galaxy.web.framework.base:132 in __call__ return self.handle_request( environ, start_response ) Module galaxy.web.framework.base:190 in handle_request body = method( trans, **kwargs ) Module galaxy.web.framework:98 in decorator return func( self, trans, *args, **kwargs ) Module galaxy.webapps.tool_shed.controllers.upload:219 in upload invalid_repository_dependencies_message = repository_dependency_util.generate_message_for_invalid_repository_dependencies( metadata_dict ) Module tool_shed.util.repository_dependency_util:252 in generate_message_for_invalid_repository_dependencies invalid_repository_dependencies = invalid_repository_dependencies_dict[ 'invalid_repository_dependencies' ] KeyError: 'invalid_repository_dependencies' extra data full traceback text version This may be an intermittent problem due to load or other unpredictable factors, reloading the page may address the problem. /quote Input repository_dependencies.xml where for the tmhmm_and_signalp entry I am missing the 'n' in name: ?xml version=1.0? repositories description=This requires my SignalP and TMHMM wrapers, and my FASTA filtering tool. !-- Revision 15:6abd809cefdd on the main tool shed is v0.2.4, the current latest - but older should be OK -- repository ame=tmhmm_and_signalp owner=peterjc / !-- Revision 2:abdd608c869b on the main tool shed is v0.0.5, the current latest - but older should be OK -- repository name=seq_filter_by_id owner=peterjc / !-- Revisiion 2:c96bef0643dc on the main tool shed is v0.0.3, the current latest -- repository name=venn_list owner=peterjc / /repositories Output repository_dependencies.xml with mangled tmhmm_and_signalp entry: ?xml version=1.0? repositories description=This requires my SignalP and TMHMM wrapers, and my FASTA filtering tool. !-- Revision 15:6abd809cefdd on the main tool shed is v0.2.4, the current latest - but older should be OK -- repository ame=tmhmm_and_signalp owner=peterjc toolshed=http://testtoolshed.g2.bx.psu.edu; / !-- Revision 2:abdd608c869b on the main tool shed is v0.0.5, the current latest - but older should be OK -- repository changeset_revision=66d1ca92fb38 name=seq_filter_by_id owner=peterjc toolshed=http://testtoolshed.g2.bx.psu.edu; / !-- Revisiion 2:c96bef0643dc on the main tool shed is v0.0.3, the current latest -- repository changeset_revision=51fe47a5a803 name=venn_list owner=peterjc toolshed=http://testtoolshed.g2.bx.psu.edu; / /repositories I corrected the ame/name error, and re-uploaded a fresh tar-ball which worked, Revision: 4:25e0ffd4f762 Peter ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified
Re: [galaxy-dev] RFC: Cheetah template in tool_dependencies.xml
Hello Björn,
Thanks very much for the patch - I've committed alsightly alterered verson of
it in changetset 10204:afec613d77fa which is currently running on the test tool
shed.
Greg Von Kuster
On Jul 7, 2013, at 6:59 PM, Björn Grüning
bjoern.gruen...@pharmazie.uni-freiburg.de wrote:
Hi Greg,
bitbucket has a downtime so I have attached a patch. I will create
tomorrow a pull request if I find time.
It will work as described and we have now a new variable (__is64bit__)
that we can use to determine if python is running under 64bit (assuming
that the rest is also 64bit-ish).
Attached is a ported version of emboss as test case (Dave B wrapper).
Have a nice evening!
Bjoern
Hi Björn,
This is great. A pull request or a patch is fine. Whatever one is
most convenient for you. The code doesn't have to be pretty, just
functional. We can always pretty it up later.
Thanks very much for doing this!
Greg
On Jul 7, 2013, at 1:19 PM, Björn Grüning
bjoern.gruen...@pharmazie.uni-freiburg.de wrote:
Hi Greg,
Hello Björn,
This would be great! I'm not sure about the type attribute - action
type=cheetah_command may be ok, but maybe a more generic type like
action type=template_command would be better.
I'm not quite sure, so go with what you feel is best.
I come up with the following:
action type=template_command
#if env.get('X11_LIB_DIR', False) and env.get('X11_INCLUDE_DIR',
False):
./configure --prefix=$env.INSTALL_DIR --x-includes=
$env.X11_INCLUDE_DIR --x-libraries=$env.X11_LIB_DIR
#else:
./configure --prefix=$env.INSTALL_DIR
#end if
/action
$env is the global access point to the env variables and its working
fine for the few repositories I tested it.
I had hoped to get it a little bit prettier but fabric and cheetah
doesn't work well together.
If that is still satisfying I would clean up my code and create a
pull request [1].
What do you think?
Cheers,
Bjoern
[1] or patch, I heard rumours that you do not like pull
requests ;)?
Thanks for looking into this!
Greg Von Kuster
On Jul 7, 2013, at 5:50 AM, Björn Grüning
bjoern.gruen...@pharmazie.uni-freiburg.de wrote:
Hi,
I was thinking how to tackle the build failures like the atlas library
on OS-X or the ones from xlib and ease the writing of the shell_command
a little bit.
Dave B did some fancy shell scripting in 'package_emboss_5_0_0' and I
was also sometimes forced to use some bash tricks.
What about to make that more easy and powerful, and as a plus more
consistent with the tool-command tag, and use cheetah for that kind of
work.
Something like that could work easily:
#if ${os.environ}.getVar('X11_LIB_DIR', None):
./configure ... --x-includes=
#else:
./configure
#end if
Also we could define some reserved variables like $__is64__ and cheetah
evaluate that. $INSTALL_DIR and $REPOSITORY_DIR can also be a cheetah
variable and we do not need to replace it later manually.
What do you think any comments?
If no one has any objections I can try to implement an action
type=cheetah_command.
Thanks,
Björn
___
Please keep all replies on the list by using reply all
in your mail client. To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
http://lists.bx.psu.edu/
To search Galaxy mailing lists use the unified search at:
http://galaxyproject.org/search/mailinglists/
tool_dependencies.xmltemplate_command.patch___
Please keep all replies on the list by using reply all
in your mail client. To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
http://lists.bx.psu.edu/
To search Galaxy mailing lists use the unified search at:
http://galaxyproject.org/search/mailinglists/
___
Please keep all replies on the list by using reply all
in your mail client. To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
http://lists.bx.psu.edu/
To search Galaxy mailing lists use the unified search at:
http://galaxyproject.org/search/mailinglists/
[galaxy-dev] Setting up reference genomes for GATK Genotyper
Hi, I am running an instance of Galaxy that I setup using Cloudman. We are trying to use the GATK Unified Genotyper, but we are missing the reference genomes. I saw in your website instructions on how to setup reference genomes for some other tools, but not for the Unified Genotyper. Where can I find instructions on how to setup reference genomes for this tool? Thanks Marco ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
Re: [galaxy-dev] Error Uploading Directory of Files
Hi Nicholas, as pointed out by Peter and here: http://www.biostars.org/p/76021/ please check your white spaces and provide a little bit more information. Btw. what is wrong with biostar? Why the double post? Cheers, Bjoern Hi, Our lab recently installed a local version of Galaxy on a mid-2012 Mac Pro computer. We can access the Galaxy server and sign in as an administrator. Today we tried creating a Data Library, adding a dataset to it, and uploading a directory of files. We followed this Galaxy documentation ( http://wiki.galaxyproject.org/Admin/DataLibraries/UploadingLibraryFiles?action=showredirect=Admin%2FData+Libraries%2FUploading+Library+Files ) to setup this feature: - Admin Data Library Add datasets Upload directory of files - file format was set to auto-detect - and we chose the option to link to files instead of copying them Galaxy confirmed that the files were successfully uploaded. However, in the data library, under the Message column, is a message in red saying Job error (click name for more info). Clicking on one of the uploaded files displays a page with this information: Date uploaded: 2013-06-28 File size: 7.5 GB Data type: auto Build: sacCer2 Miscellaneous information: Traceback (most recent call last): File /Users/administrator/galaxy-dist/tools/data_source/upload.py, line 386, in __main__() File /Users/administrator/galaxy-dist/tools/data_source/upload.py, line 357, in __main__ output_paths = Job Standard Error Traceback (most recent call last): File /Users/administrator/galaxy-dist/tools/data_source/upload.py, line 386, in __main__() File /Users/administrator/galaxy-dist/tools/data_source/upload.py, line 357, in __main__ output_paths = parse_outputs( sys.argv[4:] ) File /Users/administrator/galaxy-dist/tools/data_source/upload.py, line 64, in parse_outputs id, files_path, path = arg.split( ':', 2 ) ValueError: need more than 1 value to unpack error Database/Build: sacCer2 Number of data lines: None Disk file: /Volumes/G-SPEED Q/data/Person 2012 project/DCP2 mef.fastq Questions: 1. Should we be concerned about this error? 2. If so, what is the right way to fix it? 3. If not, how do we remove the red error message next to each file: Job error (click name for more info) ? Thank you ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/ ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
Re: [galaxy-dev] ToolShed dependencies KeyError: 'invalid_repository_dependencies'
Hello Peter, Thanks for reporting this - I have committed a fix in changeset 10206:86efa5ac1fae, which is currently running on the test tool shed. Greg Von Kuster On Jul 8, 2013, at 11:35 AM, Peter Cock p.j.a.c...@googlemail.com wrote: Hi Greg, Previously my workflows had hard coded dependencies on the main Tool Shed, which currently is a bit of a problem on the Test Tool Shed. Therefore I'm update my Tool Shed workflows using a repository_dependencies.xml with implicit repository and revision fields (i.e. the latest revision within the current ToolShed). It worked for one example but not the other - this turned out to be my own error, but the Tool Shed did not handle it gracefully. Perhaps this class of error could be caught instead? Thanks! This worked, secreted_protein_workflow, Revision: 2:aecb871dfe4c http://testtoolshed.g2.bx.psu.edu/view/peterjc/secreted_protein_workflow Input: ?xml version=1.0? repositories description=This requires my SignalP and TMHMM wrapers, and my FASTA filtering tool. !-- Revision 15:6abd809cefdd on the main tool shed is v0.2.4, the current latest - but older should be OK -- repository name=tmhmm_and_signalp owner=peterjc / !-- Revision 2:abdd608c869b on the main tool shed is v0.0.5, the current latest - but older should be OK -- repository name=seq_filter_by_id owner=peterjc / /repositories This failed, rxlr_venn_workflow, Revision: 3:65d174589251 http://testtoolshed.g2.bx.psu.edu/view/peterjc/rxlr_venn_workflow On uploading the tar-ball I got this error in the browser: quote Internal Server Error Galaxy was unable to sucessfully complete your request URL: http://testtoolshed.g2.bx.psu.edu/upload/upload?repository_id=20a9f7389ced5696 Module galaxy.web.framework.middleware.error:149 in __call__ app_iter = self.application(environ, sr_checker) Module paste.debug.prints:106 in __call__ environ, self.app) Module paste.wsgilib:543 in intercept_output app_iter = application(environ, replacement_start_response) Module paste.recursive:84 in __call__ return self.application(environ, start_response) Module paste.httpexceptions:633 in __call__ return self.application(environ, start_response) Module galaxy.web.framework.base:132 in __call__ return self.handle_request( environ, start_response ) Module galaxy.web.framework.base:190 in handle_request body = method( trans, **kwargs ) Module galaxy.web.framework:98 in decorator return func( self, trans, *args, **kwargs ) Module galaxy.webapps.tool_shed.controllers.upload:219 in upload invalid_repository_dependencies_message = repository_dependency_util.generate_message_for_invalid_repository_dependencies( metadata_dict ) Module tool_shed.util.repository_dependency_util:252 in generate_message_for_invalid_repository_dependencies invalid_repository_dependencies = invalid_repository_dependencies_dict[ 'invalid_repository_dependencies' ] KeyError: 'invalid_repository_dependencies' extra data full traceback text version This may be an intermittent problem due to load or other unpredictable factors, reloading the page may address the problem. /quote Input repository_dependencies.xml where for the tmhmm_and_signalp entry I am missing the 'n' in name: ?xml version=1.0? repositories description=This requires my SignalP and TMHMM wrapers, and my FASTA filtering tool. !-- Revision 15:6abd809cefdd on the main tool shed is v0.2.4, the current latest - but older should be OK -- repository ame=tmhmm_and_signalp owner=peterjc / !-- Revision 2:abdd608c869b on the main tool shed is v0.0.5, the current latest - but older should be OK -- repository name=seq_filter_by_id owner=peterjc / !-- Revisiion 2:c96bef0643dc on the main tool shed is v0.0.3, the current latest -- repository name=venn_list owner=peterjc / /repositories Output repository_dependencies.xml with mangled tmhmm_and_signalp entry: ?xml version=1.0? repositories description=This requires my SignalP and TMHMM wrapers, and my FASTA filtering tool. !-- Revision 15:6abd809cefdd on the main tool shed is v0.2.4, the current latest - but older should be OK -- repository ame=tmhmm_and_signalp owner=peterjc toolshed=http://testtoolshed.g2.bx.psu.edu; / !-- Revision 2:abdd608c869b on the main tool shed is v0.0.5, the current latest - but older should be OK -- repository changeset_revision=66d1ca92fb38 name=seq_filter_by_id owner=peterjc toolshed=http://testtoolshed.g2.bx.psu.edu; / !-- Revisiion 2:c96bef0643dc on the main tool shed is v0.0.3, the current latest -- repository changeset_revision=51fe47a5a803 name=venn_list owner=peterjc toolshed=http://testtoolshed.g2.bx.psu.edu; / /repositories I corrected the ame/name error, and re-uploaded a fresh tar-ball which worked, Revision:
[galaxy-dev] Connecting to new Galaxy Cloudman by FTP
Hi, I am having trouble setting up a FTP connection with the recently released version of Galaxy Cloudman (ami-118bfc78). I have instantiated the new version of Galaxy Cloudman with CloudLaunchhttp://usegalaxy.org/cloudlaunch and also through the AWS EC2 wizard (using the same security group settings as the previous versions) and neither instance will connect to my FTP connection. Has anyone else had this problem? Does anyone know what is preventing the FTP connection? Any help would be greatly appreciated. Cheers, Mo Heydarian ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
Re: [galaxy-dev] gzipped fastq reader
On Mon, Jul 8, 2013 at 10:24 PM, Robert Baertsch robert.baert...@gmail.com wrote: Peter and Dan, I like the idea of replacing all open() with galaxy_open() in all tools. You can tell the format by looking at the first 4 byes (see C code below from the UCSC browser team). Is there some pythonic way of overriding open? There is monkey patching (replace the current 'open' function with your modified version), but that is not a good idea in general. In any case, this would only affect the small number of Python tools which happen to use the Galaxy parsing libraries - which is a very small fraction of the tools in Galaxy. Most of the tools in Galaxy are compiled programs and are entirely separate. Peter ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
Re: [galaxy-dev] gzipped fastq reader
On Mon, Jul 8, 2013 at 11:21 PM, Robert Baertsch rbaer...@ucsc.edu wrote: I respectfully disagree, If you want an extensible system, you should always wrap primitive system level calls. Any tools that opens a file that could be compressed would be affected. That is a huge number of tools. Do you really want a cottage industry of tools that have different methods of dealing with compression? But defining a Python helper function within the Galaxy Python libraries doesn't achieve that. Are you talking about patching the OS level POSIX open functions or something? The tools available in Galaxy are written in a range of languages including C, Perl, R, etc. Yes, some are in Python, but of those most are independent of Galaxy and can be used separately from Galaxy. Encoding the gzip status in the datatype will create an explosion of datatypes. Compression is not actually a datatype, it tells you nothing about the content data that is stored in the file. What we'd previously discussed was a dual system, holding the file type as now (e.g. FASTA, SAM, GFF3, etc) and any compression (e.g., None, normal GZIP, BGZF which is a GZIP variant, BZIP2, etc). Galaxy tool wrappers currently define input files with a list of file types - they'd also have to give a list of supported compression types (defaulting to none). Likewise for any output files - if they are already compressed the XML for the tool wrapper would have to tell Galaxy this. It is up to the galaxy team to provide a standard way to interact with compressed files. That is my preference too - although this could be driven by the Galaxy community rather than the core team? I see defining new datatypes like 'gzippedfastq' as a stop gap special case (but a very practical route for now). My proposed solution, is a very small change that could be phased in over time. Any tools that uses open would not support compressed files, but they would not break on uncompressed files. Do others have an opinion? Either I don't understand your plan, or it would only help in a tiny minority of cases. Regards, Peter ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
