Re: [galaxy-dev] Setup galaxy to run on SGE cluster
Whoops. That's what I mean...a submission host, not an exec host. On Mon, Jan 9, 2012 at 11:13 PM, Derrick LIN wrote: > Thanks Ryan, > > I have been following that guide. I can configure the galaxy host to be an > exec host, although it makes more sense to be a submission host. > > I hope this can be clarified in the future revise of the documentation. > > Cheers, > Derrick > > > On Tue, Jan 10, 2012 at 12:57 PM, Ryan Golhar > wrote: > >> Derrick, >> >> Take a look at the section entitled "Job runner URL" on >> http://wiki.g2.bx.psu.edu/Admin/Config/Performance/Cluster. But as I >> understand it, you don't need to do anything since the drmaa library is >> part of your SGE installation. Just make the node Galaxy runs on an exec >> host. >> >> Ryan >> >> On Mon, Jan 9, 2012 at 8:45 PM, Derrick LIN wrote: >> >>> Hi guys, >>> >>> I am trying to configure our test galaxy to run on our SGE cluster >>> http://wiki.g2.bx.psu.edu/Admin/Config/Performance/Cluster. >>> >>> This is my first time to come across to drmaa, so a bit confused. >>> >>> I installed teh galaxy on a NFS location where can be accessed >>> clsuter-wide. I have configured in universe_wsgi.ini: >>> >>> start_job_runners = drmaa >>> default_cluster_job_runner = drmaa:/// >>> >>> and export the libdrmaa path: >>> export DRMAA_LIBRARY_PATH=/opt/sge/lib/lx-amd64/libdrmaa.so.1.0 >>> >>> Sound like that's all. But I am not sure how galaxy can actually bind to >>> my cluster? Anywhere I need to specify the qmaster URL etc? >>> >>> Or simply configure galaxy host to be a submission host of the cluster, >>> the galaxy will fetch all information by itself? >>> >>> Kind regards, >>> Derrick >>> >>> ___ >>> Please keep all replies on the list by using "reply all" >>> in your mail client. To manage your subscriptions to this >>> and other Galaxy lists, please use the interface at: >>> >>> http://lists.bx.psu.edu/ >>> >> >> > ___ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
Re: [galaxy-dev] Setup galaxy to run on SGE cluster
Thanks Ryan, I have been following that guide. I can configure the galaxy host to be an exec host, although it makes more sense to be a submission host. I hope this can be clarified in the future revise of the documentation. Cheers, Derrick On Tue, Jan 10, 2012 at 12:57 PM, Ryan Golhar wrote: > Derrick, > > Take a look at the section entitled "Job runner URL" on > http://wiki.g2.bx.psu.edu/Admin/Config/Performance/Cluster. But as I > understand it, you don't need to do anything since the drmaa library is > part of your SGE installation. Just make the node Galaxy runs on an exec > host. > > Ryan > > On Mon, Jan 9, 2012 at 8:45 PM, Derrick LIN wrote: > >> Hi guys, >> >> I am trying to configure our test galaxy to run on our SGE cluster >> http://wiki.g2.bx.psu.edu/Admin/Config/Performance/Cluster. >> >> This is my first time to come across to drmaa, so a bit confused. >> >> I installed teh galaxy on a NFS location where can be accessed >> clsuter-wide. I have configured in universe_wsgi.ini: >> >> start_job_runners = drmaa >> default_cluster_job_runner = drmaa:/// >> >> and export the libdrmaa path: >> export DRMAA_LIBRARY_PATH=/opt/sge/lib/lx-amd64/libdrmaa.so.1.0 >> >> Sound like that's all. But I am not sure how galaxy can actually bind to >> my cluster? Anywhere I need to specify the qmaster URL etc? >> >> Or simply configure galaxy host to be a submission host of the cluster, >> the galaxy will fetch all information by itself? >> >> Kind regards, >> Derrick >> >> ___ >> Please keep all replies on the list by using "reply all" >> in your mail client. To manage your subscriptions to this >> and other Galaxy lists, please use the interface at: >> >> http://lists.bx.psu.edu/ >> > > ___ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
Re: [galaxy-dev] Setup galaxy to run on SGE cluster
Derrick, Take a look at the section entitled "Job runner URL" on http://wiki.g2.bx.psu.edu/Admin/Config/Performance/Cluster. But as I understand it, you don't need to do anything since the drmaa library is part of your SGE installation. Just make the node Galaxy runs on an exec host. Ryan On Mon, Jan 9, 2012 at 8:45 PM, Derrick LIN wrote: > Hi guys, > > I am trying to configure our test galaxy to run on our SGE cluster > http://wiki.g2.bx.psu.edu/Admin/Config/Performance/Cluster. > > This is my first time to come across to drmaa, so a bit confused. > > I installed teh galaxy on a NFS location where can be accessed > clsuter-wide. I have configured in universe_wsgi.ini: > > start_job_runners = drmaa > default_cluster_job_runner = drmaa:/// > > and export the libdrmaa path: > export DRMAA_LIBRARY_PATH=/opt/sge/lib/lx-amd64/libdrmaa.so.1.0 > > Sound like that's all. But I am not sure how galaxy can actually bind to > my cluster? Anywhere I need to specify the qmaster URL etc? > > Or simply configure galaxy host to be a submission host of the cluster, > the galaxy will fetch all information by itself? > > Kind regards, > Derrick > > ___ > Please keep all replies on the list by using "reply all" > in your mail client. To manage your subscriptions to this > and other Galaxy lists, please use the interface at: > > http://lists.bx.psu.edu/ > ___ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
[galaxy-dev] Setup galaxy to run on SGE cluster
Hi guys, I am trying to configure our test galaxy to run on our SGE cluster http://wiki.g2.bx.psu.edu/Admin/Config/Performance/Cluster. This is my first time to come across to drmaa, so a bit confused. I installed teh galaxy on a NFS location where can be accessed clsuter-wide. I have configured in universe_wsgi.ini: start_job_runners = drmaa default_cluster_job_runner = drmaa:/// and export the libdrmaa path: export DRMAA_LIBRARY_PATH=/opt/sge/lib/lx-amd64/libdrmaa.so.1.0 Sound like that's all. But I am not sure how galaxy can actually bind to my cluster? Anywhere I need to specify the qmaster URL etc? Or simply configure galaxy host to be a submission host of the cluster, the galaxy will fetch all information by itself? Kind regards, Derrick ___ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
Re: [galaxy-dev] SAM/BAM Hybrid Selection Metrics
Hi Ryan, Yes, the Picard tool mandates a bizarre bait/target format file for reasons which might best be addressed to the Picard devs - they may have some very good reasons although I can't imagine what they are. :) Yes, automated conversion of any valid Galaxy bed dataset into the strange format required by the Picard tool is a very good idea. We're already half way there because the tool wrapper adds the (IMHO really silly) required SAM header automagically. A new datatype (eg "picardBaitTarget") and an automated converter would make the tool much easier to use - it's far from ideal to force Galaxy users to comply with the strange Picard format requirements if we can automate a converter. I thought about implementing one but stopped when I realized that am not sure what an automated converter should do if the user supplies a valid Galaxy bed lacking strand information - generally, making up strand is not a good idea. I don't have enough insight into the way the stats are calculated to know whether bad things might happen if (eg) we assume all the bait and target regions are on the + strand if they're not - but if someone can describe how to automate the conversion, it would definitely be an improvement to the usability of the Picard tool. Suggestions welcomed! On Tue, Jan 10, 2012 at 8:03 AM, Ryan Golhar wrote: > Hi all - I think there is a problem with the Picard HSMetrics wrapper in > Galaxy. The wrapper accepts a BAM files and a BED file. However the BED > file isn't really in a BED format...it requires a SAM header before the BED > lines. This really isn't a BED file format. I'm not quite sure how Galaxy > should deal with this...maybe a file format specific for Picard formatted > BED file. > > > ___ > Please keep all replies on the list by using "reply all" > in your mail client. To manage your subscriptions to this > and other Galaxy lists, please use the interface at: > > http://lists.bx.psu.edu/ -- Ross Lazarus MBBS MPH; Associate Professor, Harvard Medical School; Head, Medical Bioinformatics, BakerIDI; Tel: +61 385321444; ___ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
Re: [galaxy-dev] error on viewing workflow details via API
Glad you got it working, and thanks for the additional information. Now that I
know what's different about the non-working workflows I can probably recreate
and fix this on my end. Extracted workflows *should* function via the API.
-Dannon
On Jan 9, 2012, at 5:06 PM, Leon Mei wrote:
> Hi Dannon,
>
> Thanks for the hint. I got it working by removing "input dataset" step and
> adding it back in the workflow canvas. The previous workflow in question was
> purely extracted from history.
>
> So now I can check the details via API:
>
> ./display.py
> http://galaxy.nbic.nl/galaxy/api/workflows/e037fdb493429c2a
> Member Information
> --
> url: /galaxy/api/workflows/e037fdb493429c2a
> inputs: {'8648': {'value': '', 'label': 'Input Dataset'}}
> id: e037fdb493429c2a
> name: test
>
> I still don't understand why it is not working with an extracted workflow.
> But I can live with this work around.
>
> Cheers,
> Leon
>
>
> On Mon, Jan 9, 2012 at 3:18 PM, Dannon Baker wrote:
> Leon,
>
> I'm not seeing this behavior in any of my instances. I assume they work
> correctly through the standard galaxy interface, but is there anything
> special about the workflows in question? Is there perhaps a tool with no
> inputs in use?
>
> Thanks!
>
> -Dannon
>
>
>
>
> On Jan 6, 2012, at 11:59 AM, Leon Mei wrote:
>
> > Hi,
> >
> > I received an error when I was trying to view the details of existing
> > workflows through API.
> >
> > I can see 4 workflows owned by myself successfully. But viewing the details
> > of any workflow got me a 500 error.
> >
> > ./display.py http://galaxy.nbic.nl/galaxy/api/workflows
> > Collection Members
> > --
> > #1: /galaxy/api/workflows/e037fdb493429c2a
> > name: test
> > id: e037fdb493429c2a
> > #2: /galaxy/api/workflows/13120e62d0fbb985
> > name: imported: Partial Workflow for Student Project (1)
> > id: 13120e62d0fbb985
> > #3: /galaxy/api/workflows/c9468fdb6dc5c5f1
> > name: GAPSS SNP calling
> > id: c9468fdb6dc5c5f1
> > #4: /galaxy/api/workflows/3f5830403180d620
> > name: GAPPS CAGE
> > id: 3f5830403180d620
> >
> > 4 element(s) in collection
> >
> > ./display.py
> > http://galaxy.nbic.nl/galaxy/api/workflows/e037fdb493429c2a
> > HTTP Error 500: Internal Server Error
> > 500 Internal Server Error
> > The server has either erred or is incapable of performing
> > the requested operation.
> >
> > Here is the error message in paster.log:
> > /*
> > galaxy.web.framework ERROR 2012-01-06 17:46:53,349 Uncaught exception in
> > exposed API method:
> > Traceback (most recent call last):
> > File
> > "/opt/galaxy/prog/galaxy-2011-12-5/lib/galaxy/web/framework/__init__.py",
> > line 147, in decorator
> > return simplejson.dumps( func( self, trans, *args, **kwargs ) )
> > File "/opt/galaxy/prog/galaxy-2011-12-5/lib/galaxy/web/api/workflows.py",
> > line 68, in show
> > inputs[step.id] = {'label':step.tool_inputs['name'], 'value':""}
> > TypeError: 'NoneType' object is unsubscriptable
> > - - [06/Jan/2012:17:46:53 +0200] "GET
> > /galaxy/api/workflows/e037fdb493429c2a?key= HTTP/1.1" 500 - "-"
> > "Python-urllib/2.6"
> > */
> >
> > Any suggestion on how to solve this problem?
> >
> > Thanks,
> > Leon
> >
> > --
> > Hailiang (Leon) Mei
> > Netherlands Bioinformatics Center
> > BioAssist NGS Taskforce
> > - https://wiki.nbic.nl/index.php/Next_Generation_Sequencing
> > Skype: leon_meiMobile: +31 6 41709231
> > ___
> > Please keep all replies on the list by using "reply all"
> > in your mail client. To manage your subscriptions to this
> > and other Galaxy lists, please use the interface at:
> >
> > http://lists.bx.psu.edu/
>
>
>
>
> --
> Hailiang (Leon) Mei
> Netherlands Bioinformatics Center
> BioAssist NGS Taskforce
> - https://wiki.nbic.nl/index.php/Next_Generation_Sequencing
> Skype: leon_meiMobile: +31 6 41709231
___
Please keep all replies on the list by using "reply all"
in your mail client. To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
http://lists.bx.psu.edu/
Re: [galaxy-dev] error on viewing workflow details via API
Hi Dannon,
Thanks for the hint. I got it working by removing "input dataset" step and
adding it back in the workflow canvas. The previous workflow in question
was purely extracted from history.
So now I can check the details via API:
./display.py
http://galaxy.nbic.nl/galaxy/api/workflows/e037fdb493429c2a
Member Information
--
url: /galaxy/api/workflows/e037fdb493429c2a
inputs: {'8648': {'value': '', 'label': 'Input Dataset'}}
id: e037fdb493429c2a
name: test
I still don't understand why it is not working with an extracted workflow.
But I can live with this work around.
Cheers,
Leon
On Mon, Jan 9, 2012 at 3:18 PM, Dannon Baker wrote:
> Leon,
>
> I'm not seeing this behavior in any of my instances. I assume they work
> correctly through the standard galaxy interface, but is there anything
> special about the workflows in question? Is there perhaps a tool with no
> inputs in use?
>
> Thanks!
>
> -Dannon
>
>
>
>
> On Jan 6, 2012, at 11:59 AM, Leon Mei wrote:
>
> > Hi,
> >
> > I received an error when I was trying to view the details of existing
> workflows through API.
> >
> > I can see 4 workflows owned by myself successfully. But viewing the
> details of any workflow got me a 500 error.
> >
> > ./display.py http://galaxy.nbic.nl/galaxy/api/workflows
> > Collection Members
> > --
> > #1: /galaxy/api/workflows/e037fdb493429c2a
> > name: test
> > id: e037fdb493429c2a
> > #2: /galaxy/api/workflows/13120e62d0fbb985
> > name: imported: Partial Workflow for Student Project (1)
> > id: 13120e62d0fbb985
> > #3: /galaxy/api/workflows/c9468fdb6dc5c5f1
> > name: GAPSS SNP calling
> > id: c9468fdb6dc5c5f1
> > #4: /galaxy/api/workflows/3f5830403180d620
> > name: GAPPS CAGE
> > id: 3f5830403180d620
> >
> > 4 element(s) in collection
> >
> > ./display.py
> http://galaxy.nbic.nl/galaxy/api/workflows/e037fdb493429c2a
> > HTTP Error 500: Internal Server Error
> > 500 Internal Server Error
> > The server has either erred or is incapable of performing
> > the requested operation.
> >
> > Here is the error message in paster.log:
> > /*
> > galaxy.web.framework ERROR 2012-01-06 17:46:53,349 Uncaught exception
> in exposed API method:
> > Traceback (most recent call last):
> > File
> "/opt/galaxy/prog/galaxy-2011-12-5/lib/galaxy/web/framework/__init__.py",
> line 147, in decorator
> > return simplejson.dumps( func( self, trans, *args, **kwargs ) )
> > File
> "/opt/galaxy/prog/galaxy-2011-12-5/lib/galaxy/web/api/workflows.py", line
> 68, in show
> > inputs[step.id] = {'label':step.tool_inputs['name'], 'value':""}
> > TypeError: 'NoneType' object is unsubscriptable
> > - - [06/Jan/2012:17:46:53 +0200] "GET
> /galaxy/api/workflows/e037fdb493429c2a?key= HTTP/1.1" 500 - "-"
> "Python-urllib/2.6"
> > */
> >
> > Any suggestion on how to solve this problem?
> >
> > Thanks,
> > Leon
> >
> > --
> > Hailiang (Leon) Mei
> > Netherlands Bioinformatics Center
> > BioAssist NGS Taskforce
> > - https://wiki.nbic.nl/index.php/Next_Generation_Sequencing
> > Skype: leon_meiMobile: +31 6 41709231
> > ___
> > Please keep all replies on the list by using "reply all"
> > in your mail client. To manage your subscriptions to this
> > and other Galaxy lists, please use the interface at:
> >
> > http://lists.bx.psu.edu/
>
>
--
Hailiang (Leon) Mei
Netherlands Bioinformatics Center
BioAssist NGS Taskforce
- https://wiki.nbic.nl/index.php/Next_Generation_Sequencing
Skype: leon_meiMobile: +31 6 41709231
___
Please keep all replies on the list by using "reply all"
in your mail client. To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
http://lists.bx.psu.edu/
[galaxy-dev] SAM/BAM Hybrid Selection Metrics
Hi all - I think there is a problem with the Picard HSMetrics wrapper in Galaxy. The wrapper accepts a BAM files and a BED file. However the BED file isn't really in a BED format...it requires a SAM header before the BED lines. This really isn't a BED file format. I'm not quite sure how Galaxy should deal with this...maybe a file format specific for Picard formatted BED file. ___ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
Re: [galaxy-dev] Status on importing BAM file into Library does not update
On Fri, Jan 6, 2012 at 12:55 PM, Ryan Golhar wrote: > >> This indicates that set_meta is running locally, in the runner process. >> Can you make sure there's not a typo in your config? The other >> possibility is that external metadata setting failed and it's being retried >> internally (if that was true, you'd see messages indicated such in the >> server log). >> >> I'm pretty sure there isn't a typo. Here is anything meta related (with > comment lines removed) in my universe_wsgi.*.ini files: > > [galaxy@bic galaxy-dist]$ grep set_meta *.ini > universe_wsgi.ini:set_metadata_externally = True > universe_wsgi.runner.ini:set_metadata_externally = True > universe_wsgi.webapp.ini:set_metadata_externally = True > > [galaxy@bic galaxy-dist]$ grep meta *.ini > universe_wsgi.ini:set_metadata_externally = True > universe_wsgi.runner.ini:set_metadata_externally = True > universe_wsgi.webapp.ini:set_metadata_externally = True > > I just tried it again on some BAM files, and nothing comes up in > /var/log/messages or /var/log/httpd/error_log. Runner0.log also doesn't > show anything except for the upload job being completed. > I'm still trying to track this one down. Can I add a debug output string to show what the value of set_metadata_externally is when its read in? If so, where would I do this? ___ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
Re: [galaxy-dev] Fwd: tool restrict access
Ivan, > > >table browser > >#if $__user_email__ == "" >You are not authorized to use this tool >#else >data_source.py $output $__app__.config.output_size_limit >#end if > > > > > ... and I got the error below, which seems connected to > the cheetah syntax. Any idea of what I'm doing wrong? As is tradition in python, you need to put semi-colons after conditionals. E.g. -- #if $__user_email__ == "": You are not authorized to use this tool #else: data_source.py $output $__app__.config.output_size_limit #end if -- J. ___ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
Re: [galaxy-dev] Using workflows with steps with several inputs through the API
On Mon, Jan 9, 2012 at 9:12 AM, Dannon Baker wrote:
> Carlos,
>
> The method you describe below for mapping inputs is precisely the intended
> approach. One reason for the long identifier being used instead of a simple
> step number is that step number (ordering as you would see in the usual run
> workflow dialog) can change without it being obvious to the user. The
> ordering is based on two factors- a topological sort of the workflow graph,
> and the distance within the editor from the top left corner. In other words,
> it's possible to edit a workflow in display position only (drag the boxes
> around) and change step numbers. One approach here would be to use the step
> label ('label': 'Left-hand Reads') in your case and programmatically fetch
> and use the input id.
>
Thanks for your response, is nice to get a confirmation I'm on the
right track. I'll look into using the label.
> For your last issue-- that library dataset is actually marked as deleted on
> main, so the error is appropriate. I'm not yet sure why that is the case,
> but I'll look into it. If you retry the operation on a known non-deleted
> library dataset everything should work just fine.
>
Oh, I always try to use data from the shared libraries when testing my
workflows on main or test server, this one is from "Illumina iDEA
Datasets (sub-sampled)". I didn't realized it could have been deleted
but still show in the data library.
Thanks again,
Carlos
___
Please keep all replies on the list by using "reply all"
in your mail client. To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
http://lists.bx.psu.edu/
Re: [galaxy-dev] (no subject)
On 01/09/2012 05:04 AM, Vidya .H.K wrote: Hi, I am new user to galaxy. I have installed galaxy on the Linux local server for ChIP-seq analysis. While running MACS, an error is flagged "An error occurred running this job: /bin/sh: macs: not found" . I have set the path for macs in /bin. But still the error persists. It would be great if any one help me resolve this problem. Hi Vidya Is macs executable for the galaxy user (ie the user the galaxy server is running as) ? Regards, Hans Thanks in advance. Regards vidya ___ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ ___ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
Re: [galaxy-dev] Fwd: tool restrict access
Hi Jeremy, thank you for your answer. I followed your hint modifyng (for example) the ucsc_tablebrowser.xml as follows (not sure about the tag to have a simple error message, but this is another problem): table browser #if $__user_email__ == "" You are not authorized to use this tool #else data_source.py $output $__app__.config.output_size_limit #end if ... and I got the error below, which seems connected to the cheetah syntax. Any idea of what I'm doing wrong? Cheers, I. Traceback (most recent call last): File "/home/galaxy/galaxy-dist/lib/galaxy/jobs/runners/local.py", line 58, in run_job job_wrapper.prepare() File "/home/galaxy/galaxy-dist/lib/galaxy/jobs/__init__.py", line 411, in prepare self.command_line = self.tool.build_command_line( param_dict ) File "/home/galaxy/galaxy-dist/lib/galaxy/tools/__init__.py", line 1691, in build_command_line command_line = fill_template( self.command, context=param_dict ) File "/home/galaxy/galaxy-dist/lib/galaxy/util/template.py", line 9, in fill_template return str( Template( source=template_text, searchList=[context] ) ) File "/home/galaxy/galaxy-dist/eggs/Cheetah-2.2.2-py2.5-linux-x86_64-ucs4.egg/Cheetah/Template.py", line 1244, in __init__ self._compile(source, file, compilerSettings=compilerSettings) File "/home/galaxy/galaxy-dist/eggs/Cheetah-2.2.2-py2.5-linux-x86_64-ucs4.egg/Cheetah/Template.py", line 1538, in _compile keepRefToGeneratedCode=True) File "/home/galaxy/galaxy-dist/eggs/Cheetah-2.2.2-py2.5-linux-x86_64-ucs4.egg/Cheetah/Template.py", line 745, in compile compiler.compile() File "/home/galaxy/galaxy-dist/eggs/Cheetah-2.2.2-py2.5-linux-x86_64-ucs4.egg/Cheetah/Compiler.py", line 1670, in compile self._parser.parse() File "/home/galaxy/galaxy-dist/eggs/Cheetah-2.2.2-py2.5-linux-x86_64-ucs4.egg/Cheetah/Parser.py", line 1496, in parse self.assertEmptyOpenDirectivesStack() File "/home/galaxy/galaxy-dist/eggs/Cheetah-2.2.2-py2.5-linux-x86_64-ucs4.egg/Cheetah/Parser.py", line 2658, in assertEmptyOpenDirectivesStack raise ParseError(self, msg=errorMsg) ParseError: Some #directives are missing their corresponding #end ___ tag: if Line 2, column 8 Line|Cheetah Code |- 2 | ^ Ivan, #if $__user_email__ == "" You are not authorized to use this tool #else data_source.py $output $__app__.config.output_size_limit To make this approach work, the email check should go in the command tag. More information: If you're looking to require users to login before using any tools, you can use this flag in the universe config file: # Force everyone to log in (disable anonymous access). #require_login = False If you're looking to implement tool-based access control, the best approach is probably to use the same role-based approach that libraries use: https://bitbucket.org/galaxy/galaxy-central/issue/269/use-galaxy-security-to-restrict-tool Thanks, J. ___ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
[galaxy-dev] Python/R Script Not Integrating with Galaxy Correctly
Hello Galaxy-Dev List,
I am having some issues in getting a script to work correctly in Galaxy. My
script is written in Python, uses RPy2 to pass some data to the R environment
and then calls a custom R function; the output of this function, and of the
greater Python/R tool, is a file. I am having 2 problems: the first is that R
cannot find the R file holding the function, and the second is that Galaxy
cannot find the output file. Before I elaborate on these issues, I should
point out that when called from the command line this script executes exactly
as I intend it to.
First issue:
When called by Galaxy, the Python portion of the script runs fine, and the RPy2
portion devoted to setting up the R environment so that the script can run
seems to execute just fine. The script fails when it tells R to call the
custom function, with the following output:
Python output:
Traceback (most recent call last):
File
"/Users/kipperfletez-brant/galaxy-dist/tools/myTools/svm_enh/svm_train_classify_cv.py",
line 865, in
if __name__=='__main__': main()
File
"/Users/kipperfletez-brant/galaxy-dist/tools/myTools/svm_enh/svm_train_classify_cv.py",
line 851, in main
robjects.r.source(current)
File
"/opt/local/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/site-packages/rpy2/robjects/functions.py",
line 82, in __call__
return super(SignatureTranslatedFunction, self).__call__(*args, **kwargs)
File
"/opt/local/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/site-packages/rpy2/robjects/functions.py",
line 34, in __call__
res = super(Function, self).__call__(*new_args, **new_kwargs)
rpy2.rinterface.RRuntimeError: Error in file(file, "r", encoding = encoding) :
cannot open the connection
R output:
/Users/kipperfletez-brant/galaxy-dist/database/job_working_directory/106/rocprc.R
[1] "/Users/kipperfletez-brant/galaxy-dist/database/job_working_directory/106"
Error in file(file, "r", encoding = encoding) :
cannot open the connection
In addition: Warning message:
In file(file, "r", encoding = encoding) :
cannot open file
'/Users/kipperfletez-brant/galaxy-dist/database/job_working_directory/106/rocprc.R':
No such file or directory
The first line of the R output is the return of a call to the Python function
os.path.abspath('rocprc.R') (note that os.path.realpath('rocprc.R') returns the
same path), while the second line, for comparison, is a call to R's getwd() -
the R file is in the same directory as the R environment's setting, but for
some reason R cannot find the file. Because this works from the command line,
I suspect that I am not taking some Galaxy setting into account.
Second issue:
Trying to identify even the kind of error, I hard-coded the directory in which
I know the R file to reside in a call to R's setwd(). This time, the script
runs correctly, but there is no file returned by Galaxy. The script execution
ends with the following output message:
empty
format: pdf, database: ?
Info: [1] "/Users/kipperfletez-brant/galaxy-dist/tools/myTools/svm_enh"
Watching the file in which the R script is held, I know that the output PDF is
saved in the same file as that specified in my tool's XML, which, for other
tools I've written, has resulted in an output file. Looking at the log, the
following DEBUG message is written:
galaxy.jobs DEBUG 2012-01-09 10:16:49,899 finish(): Could not move
/Users/kipperfletez-brant/galaxy-dist/database/job_working_directory/111/rocprc.pdf
to /Users/kipperfletez-brant/galaxy-dist/database/files/000/dataset_111.dat as
directed by from_work_dir
As with the first issue, I suspect there is some sort of Galaxy setting I am
unaware of.
Generally, because both issues relate to finding files not directly
created/referenced by the caller Python script, I suspect that there will be
some commonality between the resolutions of the two issues, but I am need help
to see that.
Thank you for reading,
Kipper Fletez-Brant
___
Please keep all replies on the list by using "reply all"
in your mail client. To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
http://lists.bx.psu.edu/
Re: [galaxy-dev] error on viewing workflow details via API
Leon,
I'm not seeing this behavior in any of my instances. I assume they work
correctly through the standard galaxy interface, but is there anything special
about the workflows in question? Is there perhaps a tool with no inputs in use?
Thanks!
-Dannon
On Jan 6, 2012, at 11:59 AM, Leon Mei wrote:
> Hi,
>
> I received an error when I was trying to view the details of existing
> workflows through API.
>
> I can see 4 workflows owned by myself successfully. But viewing the details
> of any workflow got me a 500 error.
>
> ./display.py http://galaxy.nbic.nl/galaxy/api/workflows
> Collection Members
> --
> #1: /galaxy/api/workflows/e037fdb493429c2a
> name: test
> id: e037fdb493429c2a
> #2: /galaxy/api/workflows/13120e62d0fbb985
> name: imported: Partial Workflow for Student Project (1)
> id: 13120e62d0fbb985
> #3: /galaxy/api/workflows/c9468fdb6dc5c5f1
> name: GAPSS SNP calling
> id: c9468fdb6dc5c5f1
> #4: /galaxy/api/workflows/3f5830403180d620
> name: GAPPS CAGE
> id: 3f5830403180d620
>
> 4 element(s) in collection
>
> ./display.py
> http://galaxy.nbic.nl/galaxy/api/workflows/e037fdb493429c2a
> HTTP Error 500: Internal Server Error
> 500 Internal Server Error
> The server has either erred or is incapable of performing
> the requested operation.
>
> Here is the error message in paster.log:
> /*
> galaxy.web.framework ERROR 2012-01-06 17:46:53,349 Uncaught exception in
> exposed API method:
> Traceback (most recent call last):
> File
> "/opt/galaxy/prog/galaxy-2011-12-5/lib/galaxy/web/framework/__init__.py",
> line 147, in decorator
> return simplejson.dumps( func( self, trans, *args, **kwargs ) )
> File "/opt/galaxy/prog/galaxy-2011-12-5/lib/galaxy/web/api/workflows.py",
> line 68, in show
> inputs[step.id] = {'label':step.tool_inputs['name'], 'value':""}
> TypeError: 'NoneType' object is unsubscriptable
> - - [06/Jan/2012:17:46:53 +0200] "GET
> /galaxy/api/workflows/e037fdb493429c2a?key= HTTP/1.1" 500 - "-"
> "Python-urllib/2.6"
> */
>
> Any suggestion on how to solve this problem?
>
> Thanks,
> Leon
>
> --
> Hailiang (Leon) Mei
> Netherlands Bioinformatics Center
> BioAssist NGS Taskforce
> - https://wiki.nbic.nl/index.php/Next_Generation_Sequencing
> Skype: leon_meiMobile: +31 6 41709231
> ___
> Please keep all replies on the list by using "reply all"
> in your mail client. To manage your subscriptions to this
> and other Galaxy lists, please use the interface at:
>
> http://lists.bx.psu.edu/
___
Please keep all replies on the list by using "reply all"
in your mail client. To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
http://lists.bx.psu.edu/
Re: [galaxy-dev] Using workflows with steps with several inputs through the API
Carlos,
The method you describe below for mapping inputs is precisely the intended
approach. One reason for the long identifier being used instead of a simple
step number is that step number (ordering as you would see in the usual run
workflow dialog) can change without it being obvious to the user. The ordering
is based on two factors- a topological sort of the workflow graph, and the
distance within the editor from the top left corner. In other words, it's
possible to edit a workflow in display position only (drag the boxes around)
and change step numbers. One approach here would be to use the step label
('label': 'Left-hand Reads') in your case and programmatically fetch and use
the input id.
For your last issue-- that library dataset is actually marked as deleted on
main, so the error is appropriate. I'm not yet sure why that is the case, but
I'll look into it. If you retry the operation on a known non-deleted library
dataset everything should work just fine.
Thanks!
Dannon
On Jan 5, 2012, at 5:35 PM, Carlos Borroto wrote:
> Hi,
>
> Looking at workflow_execute.py, it seems to me there can be only one
> input for step, is this correct? or is there a wait to map more than
> one input and match them to an specific position in a particularly
> step?
>
> This is the workaround I found.
>
> I created 'Input Dataset' boxes for all of my inputs, then I linked
> them to the step where they are needed. This is actually very
> convenient if the same input is used in more than one step, as when I
> run the workflow manually I only need to select each input once. I
> tried doing this, as each 'Input Dataset' would have his own step.
>
> This is an example workflow:
> http://main.g2.bx.psu.edu/u/cjav/w/fastq-joiner---input-datasets
>
> Initially I ran into this issue:
> $ ./workflow_execute.py my_key http://main.g2.bx.psu.edu/api/workflows
> fcfa9ae643d37c32 'Fastq joiner - Input Datasets'
> '1=ldda=b4c5be159c8057c2' '2=ldda=ea0e32961d454539'
> HTTP Error 400: Bad Request
> "Workflow cannot be run because an expected input step '517620' has no
> input dataset."
>
> I then found you can do:
> $ ./display.py my_key http://main.g2.bx.psu.edu/api/workflows/fcfa9ae643d37c32
> Member Information
> --
> url: /api/workflows/fcfa9ae643d37c32
> inputs: {'517620': {'value': '', 'label': 'Left-hand Reads'},
> '517622': {'value': '', 'label': 'Right-hand Reads'}}
> id: fcfa9ae643d37c32
> name: Fastq joiner - Input Datasets
>
>
> I see inputs get different keys and I can use them when executing the
> workflow:
> $ ./workflow_execute.py my_key http://main.g2.bx.psu.edu/api/workflows
> fcfa9ae643d37c32 'hist_id=9fbfd3a1be8b7f8e'
> '517620=hda=8d89405199368bb0' '517622=hda=5c93f30078e5ac25'
> Response
>
> {'outputs': ['f7c4b0900d8f3b1b'], 'history': '9fbfd3a1be8b7f8e'}
>
> This worked perfectly. I still think it would be easier to be able to
> use the step number, but this works.
>
>
> Lastly, I did run into one issue where I had to import the datasets
> firsts, as trying to use ldda like this:
> $ ./workflow_execute.py my_key http://main.g2.bx.psu.edu/api/workflows
> fcfa9ae643d37c32 'Fastq joiner - Input Datasets'
> '517620=ldda=b4c5be159c8057c2' '517622=ldda=ea0e32961d454539'
> Response
>
> {'outputs': ['44c033fad737acc5'], 'history': '9fbfd3a1be8b7f8e'}
>
> I got this error in the history:
> input data 1 (file:
> /galaxy/main_database/files/002/780/dataset_2780871.dat) was deleted
> before the job started
>
> This didn't happen in our local instance using galaxy-central.
>
> Thanks,
> Carlos
> ___
> Please keep all replies on the list by using "reply all"
> in your mail client. To manage your subscriptions to this
> and other Galaxy lists, please use the interface at:
>
> http://lists.bx.psu.edu/
___
Please keep all replies on the list by using "reply all"
in your mail client. To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
http://lists.bx.psu.edu/
Re: [galaxy-dev] broken pipe when running VCF-merge.
Hi,
We are attempting to integrate vcftools specifically vcf-merge into
galaxy. We have two or files that we
are attempting to merge, the command when run through terminal does not
give a broken pipe, but
when galaxy runs the command a broken pipe is given at the beginning and
the start of the execution.
The output from the merge appears to be exactly the same. But the broken
pipe error causes galaxy
to go read on the output wondering if anyone can help that would be much
appreciated.
The shell script is pasted below.
_SHELL SCRIPT THAT GALAXY RUNS
_
1 #!/bin/bash
2 # @Date 9/01/2012
3 # @Author Ed Hills, James Boocock
4 #
5 # This file will take a white space separated list of file names,
6 # and run the vcf-merge tool and merge all files into a new file.
7 #
8 # INPUTS
9 # $1 = First input File
10 # $2 = Second input File
11 # $N = Extra input files
12
13 FILE_LIST=""
14
15 cat $1 | bgzip -c > ~tmp1.vcf.gz
16 cat $2 | bgzip -c > ~tmp2.vcf.gz
17
18 tabix -p vcf ~tmp1.vcf.gz
19 tabix -p vcf ~tmp2.vcf.gz
20
21 FILE_LIST="~tmp1.vcf.gz ~tmp2.vcf.gz"
22
23 if [ $# > 2 ]
24 then
25 for ((i=3; i <= $#; i++))
26 do
27 eval EXTRA_FILE=\$${i}
28 cat $EXTRA_FILE | bgzip -c > ~tmp${i}.vcf.gz
29 tabix -p vcf ~tmp${i}.vcf.gz
30 FILE_LIST="${FILE_LIST} ~tmp${i}.vcf.gz"
31 done
32 fi
33
34 perl ~/galaxy-dist/tool-data/shared/vcfperltools/vcf-merge
${FILE_LIST} 2> /dev/null
35
36 rm -f ~tmp*
37
_ERROR MESSAGE_
Warning: The AC tag(s) will not be merged correctly for multiallelic sites.
To be handled correctly, please redefine with Number=. or set the environment
variable DONT_FIX_VCF40_AG_TAGS=0.
gzip: stdout: Broken pipe
Using column name 'HG00096' for ~tmp1.vcf.gz:HG00096
gzip: stdout: Broken pipe
Using column name 'NA06986' for ~tmp2.vcf.gz:NA06986
Cheers James.
___
Please keep all replies on the list by using "reply all"
in your mail client. To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
http://lists.bx.psu.edu/
[galaxy-dev] (no subject)
Hi, I am new user to galaxy. I have installed galaxy on the Linux local server for ChIP-seq analysis. While running MACS, an error is flagged "An error occurred running this job: /bin/sh: macs: not found" . I have set the path for macs in /bin. But still the error persists. It would be great if any one help me resolve this problem. Thanks in advance. Regards vidya ___ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
