Re: [galaxy-dev] Is the new tool repositories summary in the monthly newsletter useful?
Hi Dave, on IRC we had a small discussion how we can make the progress easier for you to collect this information. The idea was to create automatically Trello cards for IUC and devteam tools and maybe a summary page in the Tool Shed. Cheers, Bjoern Am 15.10.2014 um 06:52 schrieb Dave Clements: Hello all, Well, the results are unanimous: Keep it; it's worth the time invested Where to put it was far from unanimous, so we'll experiment with putting it in both the monthly newsletter and dev news briefs. Thanks all, Dave C. On Wed, Oct 8, 2014 at 9:42 AM, Dave Clements cleme...@galaxyproject.org wrote: Hi Peter, all, I've added post in both places as an option. So far we only have two responses ... Thanks, Dave C On Wed, Oct 8, 2014 at 1:16 AM, Peter Cock p.j.a.c...@googlemail.com wrote: On Wed, Oct 8, 2014 at 12:49 AM, Dave Clements cleme...@galaxyproject.org wrote: Hi All, The October Galaxy newsletter went out a week ago. Buried at the bottom is this 36 new ToolShed repos -- https://wiki.galaxyproject.org/GalaxyUpdates/2014_10#ToolShed_Contributions which lists repositories that have been published in the Galaxy Project ToolShed in the previous month. I have two questions about this: 1. How useful is this summary? Compiling it is a manual process and it's kind of mind-numbing. Most months it takes around 2 hours (I think). I find it moderately useful, so if most Galaxy Admins think the same, it probably is overall a good time investment. 2. If we keep the summary, should we put it in the Dev News Briefs instead? I'm kinda thinking this summary is a better match for the Dev News Briefs (every release), then it is for the general newsletter (every month). I would suggest both (easy if it is just a link, a tiny bit of copy and paste if not), but that wasn't an option on the Google form. Peter -- http://galaxyproject.org/ http://getgalaxy.org/ http://usegalaxy.org/ https://wiki.galaxyproject.org/ ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/ ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
[galaxy-dev] Set output dbkey from parameter value
Hi all,
I'm trying to set the dbkey of an output file from the value (text) of
a parameter.
The parameter I want to use is genome.
conditional name=bedgraph
param name=check type=select label=Produce BedGraph
output help=Can be displayed directly on UCSC browser. One file per
normalisation method.
option value=no selected=TrueNo/option
option value=yesYes/option
/param
when value=yes
param name=bed_file type=data format=bed
label=Transcripts ins BED format help=12 column BED file containing
trancript definitions. /
param name=genome type=text label=Genome Build
help=E.g. hg19 /
param name=track_name type=text label=Track
Name size=20 value=Track Name /
/when
when value=no /
/conditional
and this is how I've set the output:
data format=bedgraph name=bedgraph_slograt
label=${tool.name} on ${on_string}: Smoot Log2ratio (bedGraph)
from_work_dir=output_dir/slograt.bedgraph
filter bedgraph['check'] == 'yes' and slograt['check']
== 'yes' /filter
action type=metadata name=dbkey
option type=from_param name=bedgraph.genome
param_attribute=value /
/action
/data
When I run the tool the dbkey isn't set to the output file. Does
anyone know a workaround?
Bests,
Nikos
___
Please keep all replies on the list by using reply all
in your mail client. To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
http://lists.bx.psu.edu/
To search Galaxy mailing lists use the unified search at:
http://galaxyproject.org/search/mailinglists/
Re: [galaxy-dev] UCSC Main table browser
I believe there is a fix in the latest update to Galaxy for this thanks to Nate. https://bitbucket.org/galaxy/galaxy-central/commits/579d211 The previous change to that file was in 2011 - so you are likely to be able to apply the commit to any release of Galaxy you may be running. -John On Tue, Oct 14, 2014 at 4:41 PM, Shrum, Donald C dcsh...@admin.fsu.edu wrote: When I click get output in the UCSC main table browser in firefox nothing happens. I can fix this by going to about:config in firefox and disabling security.mixed_content.block_active_content I am running galaxy behind an apache proxy using https as we have users authenticate with their hpc accounts. Is there a fix for this aside from making the change to firefox? Thanks, Don FSU Research Computing Center ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/ ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
Re: [galaxy-dev] snpeff tool for Galaxy extra_files_path
JJ, Arg this is a mess. I am very sorry about this - I still don't understand extra_files_path versus files_path myself. There are open questions on Peter's blast repo and no one ever followed up on my object store questions about this with Bjoern's issues a couple release cycles ago. We need to get these to work - write documetation explicitly declaring best practices we can all agree on and then write some tests to ensure things don't break in the future. When you say your tools broke recently - can you say for certain which release broke these - the August14, October14, something older? I'll try to do some more research and get back to you. -John On Tue, Oct 14, 2014 at 6:04 AM, Jim Johnson johns...@umn.edu wrote: Andrew, Thanks for investigating this. I changed the subject and sent to the galaxy dev list. I've had a number of tools quit working recently. Particularly tools that inspect the extra_files_path when setting metadata, Defuse, Rsem, SnpEff. I think there was a change in the galaxy framework: The extra_files_path when referenced from an input or output in the cheetah template sections of the tool config xml will be relative to the job working directly rather than the files location. I've just changed a few of my tools on my server yesterday from: param_name.extra_files_path to: param_name.dataset.extra_files_path and they now work again. Dan or John, is that the right way to handle this? Thanks, JJ On 10/13/14, 9:29 PM, Andrew Lonie wrote: Hi Jim. I am probably going about this the wrong way, but I am not clear on how to report tool errors (if in fact this is a tool error!) I've been trialling your snpeff wrapper from the test toolshed and getting a consistent error with the SnpEff Download and SnpEff sub tools (the SnpSift dbNSFP works fine). The problem seems to be with an attribute declaration and manifests during database download as: Traceback (most recent call last): File /mnt/galaxy/galaxy-app/lib/galaxy/jobs/runners/__init__.py, line 564, in finish_job job_state.job_wrapper.finish( stdout, stderr, exit_code ) File /mnt/galaxy/galaxy-app/lib/galaxy/jobs/__init__.py, line 1107, in finish dataset.datatype.set_meta( dataset, overwrite=False ) # call datatype.set_meta directly for the initial set_meta call during dataset creation File /mnt/galaxy/shed_tools/testtoolshed.g2.bx.psu.edu/repos/iuc/snpeff/1938721334b3/snpeff/lib/galaxy/datatypes/snpeff.py, line 21, in set_meta data_dir = dataset.files_path AttributeError: 'HistoryDatasetAssociation' object has no attribute 'files_path' We fiddled around with the wrapper, eventually replacing 'dataset.files_path' with 'dataset.extra_files_path' in snpeff.py, which fixed the download bug, but then SnpEff subtool itself threw a similar error when I tried to use that database from the history. I chased up a bit more but cannot understand the various posts on files_path vs extra_files_path I've shared a history with both of these errors here: http://130.56.251.62/galaxy/u/alonie/h/unnamed-history Maybe this is a problem with our Galaxy image? Any help appreciated! Andrew A/Prof Andrew Lonie University of Melbourne -- James E. Johnson Minnesota Supercomputing Institute University of Minnesota ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/ ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
Re: [galaxy-dev] snpeff tool for Galaxy extra_files_path
Okay - so this is what broke things: https://bitbucket.org/galaxy/galaxy-central/commits/d781366bc120787e201b73a4dd99b56282169d86 My feeling with the commit was that wrappers and tools should never be explicitly accessing paths explicitly through input.dataset.*. I think this would circumvent options like outputs_to_working_directory and break remote job execution through Pulsar. It also breaks the object store abstraction I think - which is why I made the change for Bjoern I guess. I did not (and this was stupid on my part) realize that datatype code would be running on the remote host and accessing these model properties directly outside the abstractions setup by the wrappers supplied to cheetah code and so they have become out of sync as of that commit. I am thinking somehow changing what the datatype code gets is the right approach and not fixing things by circumvent the wrapper and accessing properties directly on the dataset. Since you will find that doing this breaks things for Bjoern object store and could probably never run on usegalaxy.org say for the same reason. Too many different competing deployment options all being incompatible with each other :(. Will keep thinking about this and respond again. -John On Wed, Oct 15, 2014 at 9:39 AM, John Chilton jmchil...@gmail.com wrote: JJ, Arg this is a mess. I am very sorry about this - I still don't understand extra_files_path versus files_path myself. There are open questions on Peter's blast repo and no one ever followed up on my object store questions about this with Bjoern's issues a couple release cycles ago. We need to get these to work - write documetation explicitly declaring best practices we can all agree on and then write some tests to ensure things don't break in the future. When you say your tools broke recently - can you say for certain which release broke these - the August14, October14, something older? I'll try to do some more research and get back to you. -John On Tue, Oct 14, 2014 at 6:04 AM, Jim Johnson johns...@umn.edu wrote: Andrew, Thanks for investigating this. I changed the subject and sent to the galaxy dev list. I've had a number of tools quit working recently. Particularly tools that inspect the extra_files_path when setting metadata, Defuse, Rsem, SnpEff. I think there was a change in the galaxy framework: The extra_files_path when referenced from an input or output in the cheetah template sections of the tool config xml will be relative to the job working directly rather than the files location. I've just changed a few of my tools on my server yesterday from: param_name.extra_files_path to: param_name.dataset.extra_files_path and they now work again. Dan or John, is that the right way to handle this? Thanks, JJ On 10/13/14, 9:29 PM, Andrew Lonie wrote: Hi Jim. I am probably going about this the wrong way, but I am not clear on how to report tool errors (if in fact this is a tool error!) I've been trialling your snpeff wrapper from the test toolshed and getting a consistent error with the SnpEff Download and SnpEff sub tools (the SnpSift dbNSFP works fine). The problem seems to be with an attribute declaration and manifests during database download as: Traceback (most recent call last): File /mnt/galaxy/galaxy-app/lib/galaxy/jobs/runners/__init__.py, line 564, in finish_job job_state.job_wrapper.finish( stdout, stderr, exit_code ) File /mnt/galaxy/galaxy-app/lib/galaxy/jobs/__init__.py, line 1107, in finish dataset.datatype.set_meta( dataset, overwrite=False ) # call datatype.set_meta directly for the initial set_meta call during dataset creation File /mnt/galaxy/shed_tools/testtoolshed.g2.bx.psu.edu/repos/iuc/snpeff/1938721334b3/snpeff/lib/galaxy/datatypes/snpeff.py, line 21, in set_meta data_dir = dataset.files_path AttributeError: 'HistoryDatasetAssociation' object has no attribute 'files_path' We fiddled around with the wrapper, eventually replacing 'dataset.files_path' with 'dataset.extra_files_path' in snpeff.py, which fixed the download bug, but then SnpEff subtool itself threw a similar error when I tried to use that database from the history. I chased up a bit more but cannot understand the various posts on files_path vs extra_files_path I've shared a history with both of these errors here: http://130.56.251.62/galaxy/u/alonie/h/unnamed-history Maybe this is a problem with our Galaxy image? Any help appreciated! Andrew A/Prof Andrew Lonie University of Melbourne -- James E. Johnson Minnesota Supercomputing Institute University of Minnesota ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at:
Re: [galaxy-dev] Set output dbkey from parameter value
Hi Nikos,
In the very least, you’ll want to make sure that you have a bounding
actions/actions tag set around your actions. It is probably also advisable
to add a set of conditional/whens around the action, since you’re only setting
the dbkey under certain circumstances.
Thanks for using Galaxy,
Dan
On Oct 15, 2014, at 6:37 AM, Nikos Sidiropoulos nikos.sid...@gmail.com wrote:
Hi all,
I'm trying to set the dbkey of an output file from the value (text) of
a parameter.
The parameter I want to use is genome.
conditional name=bedgraph
param name=check type=select label=Produce BedGraph
output help=Can be displayed directly on UCSC browser. One file per
normalisation method.
option value=no selected=TrueNo/option
option value=yesYes/option
/param
when value=yes
param name=bed_file type=data format=bed
label=Transcripts ins BED format help=12 column BED file containing
trancript definitions. /
param name=genome type=text label=Genome Build
help=E.g. hg19 /
param name=track_name type=text label=Track
Name size=20 value=Track Name /
/when
when value=no /
/conditional
and this is how I've set the output:
data format=bedgraph name=bedgraph_slograt
label=${tool.name} on ${on_string}: Smoot Log2ratio (bedGraph)
from_work_dir=output_dir/slograt.bedgraph
filter bedgraph['check'] == 'yes' and slograt['check']
== 'yes' /filter
action type=metadata name=dbkey
option type=from_param name=bedgraph.genome
param_attribute=value /
/action
/data
When I run the tool the dbkey isn't set to the output file. Does
anyone know a workaround?
Bests,
Nikos
___
Please keep all replies on the list by using reply all
in your mail client. To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
http://lists.bx.psu.edu/
To search Galaxy mailing lists use the unified search at:
http://galaxyproject.org/search/mailinglists/
___
Please keep all replies on the list by using reply all
in your mail client. To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
http://lists.bx.psu.edu/
To search Galaxy mailing lists use the unified search at:
http://galaxyproject.org/search/mailinglists/
[galaxy-dev] CloudMan auto-scaling - Use spot instance for worker node?
Hi all, I realize when you manually add a node you can request that it be a spot instance. Is it also possible to use spot instances for auto-scaling? Thank you very much, Robert This e-mail message (including any attachments) is for the sole use of the intended recipient(s) and may contain confidential and privileged information. If the reader of this message is not the intended recipient, you are hereby notified that any dissemination, distribution or copying of this message (including any attachments) is strictly prohibited. If you have received this message in error, please contact the sender by reply e-mail message and destroy all copies of the original message (including attachments). ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
Re: [galaxy-dev] Set output dbkey from parameter value
Hi Daniel,
Thanks for the response.
I've edited the output to:
data format=bedgraph name=bedgraph_slograt
label=${tool.name} on ${on_string}: Smoot Log2ratio (bedGraph)
from_work_dir=output_dir/slograt.bedgraph
filter bedgraph['check'] == 'yes' and slograt['check']
== 'yes' /filter
actions
conditional name=bedgraph.check
when value=yes
action type=metadata name=dbkey
option type=from_param
name=bedgraph.genome param_attribute=value /
/action
/when
/conditional
/actions
/data
Now I'm getting a tool execution error.
Error executing tool: 'unicode' object has no attribute 'value'
I've tried to change the param_attribute to ext, dbkey (ones that
I know that exist) and got a similar error.
Bests,
Nikos
On 15 October 2014 16:58, Daniel Blankenberg d...@bx.psu.edu wrote:
Hi Nikos,
In the very least, you’ll want to make sure that you have a bounding
actions/actions tag set around your actions. It is probably also
advisable to add a set of conditional/whens around the action, since you’re
only setting the dbkey under certain circumstances.
Thanks for using Galaxy,
Dan
On Oct 15, 2014, at 6:37 AM, Nikos Sidiropoulos nikos.sid...@gmail.com
wrote:
Hi all,
I'm trying to set the dbkey of an output file from the value (text) of
a parameter.
The parameter I want to use is genome.
conditional name=bedgraph
param name=check type=select label=Produce BedGraph
output help=Can be displayed directly on UCSC browser. One file per
normalisation method.
option value=no selected=TrueNo/option
option value=yesYes/option
/param
when value=yes
param name=bed_file type=data format=bed
label=Transcripts ins BED format help=12 column BED file containing
trancript definitions. /
param name=genome type=text label=Genome Build
help=E.g. hg19 /
param name=track_name type=text label=Track
Name size=20 value=Track Name /
/when
when value=no /
/conditional
and this is how I've set the output:
data format=bedgraph name=bedgraph_slograt
label=${tool.name} on ${on_string}: Smoot Log2ratio (bedGraph)
from_work_dir=output_dir/slograt.bedgraph
filter bedgraph['check'] == 'yes' and slograt['check']
== 'yes' /filter
action type=metadata name=dbkey
option type=from_param name=bedgraph.genome
param_attribute=value /
/action
/data
When I run the tool the dbkey isn't set to the output file. Does
anyone know a workaround?
Bests,
Nikos
___
Please keep all replies on the list by using reply all
in your mail client. To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
http://lists.bx.psu.edu/
To search Galaxy mailing lists use the unified search at:
http://galaxyproject.org/search/mailinglists/
___
Please keep all replies on the list by using reply all
in your mail client. To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
http://lists.bx.psu.edu/
To search Galaxy mailing lists use the unified search at:
http://galaxyproject.org/search/mailinglists/
Re: [galaxy-dev] snpeff tool for Galaxy extra_files_path
I agree with you about the inadvisable use of: input.dataset.*. I'm looking at: lib/galaxy/model/__init__.py class Dataset( object ): ... def __init__( self, id=None, state=None, external_filename=None, extra_files_path=None, file_size=None, purgable=True, uuid=None ): ... self._extra_files_path = extra_files_path ... @property def extra_files_path( self ): return self.object_store.get_filename( self, dir_only=True, extra_dir=self._extra_files_path or dataset_%d_files % self.id ) I'm trying to see when self._extra_files_path gets set. Otherwise, would this return the path relative to the current file location of dataset? On 10/15/14, 9:36 AM, John Chilton wrote: Okay - so this is what broke things: https://bitbucket.org/galaxy/galaxy-central/commits/d781366bc120787e201b73a4dd99b56282169d86 My feeling with the commit was that wrappers and tools should never be explicitly accessing paths explicitly through input.dataset.*. I think this would circumvent options like outputs_to_working_directory and break remote job execution through Pulsar. It also breaks the object store abstraction I think - which is why I made the change for Bjoern I guess. I did not (and this was stupid on my part) realize that datatype code would be running on the remote host and accessing these model properties directly outside the abstractions setup by the wrappers supplied to cheetah code and so they have become out of sync as of that commit. I am thinking somehow changing what the datatype code gets is the right approach and not fixing things by circumvent the wrapper and accessing properties directly on the dataset. Since you will find that doing this breaks things for Bjoern object store and could probably never run on usegalaxy.org say for the same reason. Too many different competing deployment options all being incompatible with each other :(. Will keep thinking about this and respond again. -John On Wed, Oct 15, 2014 at 9:39 AM, John Chilton jmchil...@gmail.com wrote: JJ, Arg this is a mess. I am very sorry about this - I still don't understand extra_files_path versus files_path myself. There are open questions on Peter's blast repo and no one ever followed up on my object store questions about this with Bjoern's issues a couple release cycles ago. We need to get these to work - write documetation explicitly declaring best practices we can all agree on and then write some tests to ensure things don't break in the future. When you say your tools broke recently - can you say for certain which release broke these - the August14, October14, something older? I'll try to do some more research and get back to you. -John On Tue, Oct 14, 2014 at 6:04 AM, Jim Johnson johns...@umn.edu wrote: Andrew, Thanks for investigating this. I changed the subject and sent to the galaxy dev list. I've had a number of tools quit working recently. Particularly tools that inspect the extra_files_path when setting metadata, Defuse, Rsem, SnpEff. I think there was a change in the galaxy framework: The extra_files_path when referenced from an input or output in the cheetah template sections of the tool config xml will be relative to the job working directly rather than the files location. I've just changed a few of my tools on my server yesterday from: param_name.extra_files_path to: param_name.dataset.extra_files_path and they now work again. Dan or John, is that the right way to handle this? Thanks, JJ On 10/13/14, 9:29 PM, Andrew Lonie wrote: Hi Jim. I am probably going about this the wrong way, but I am not clear on how to report tool errors (if in fact this is a tool error!) I've been trialling your snpeff wrapper from the test toolshed and getting a consistent error with the SnpEff Download and SnpEff sub tools (the SnpSift dbNSFP works fine). The problem seems to be with an attribute declaration and manifests during database download as: Traceback (most recent call last): File /mnt/galaxy/galaxy-app/lib/galaxy/jobs/runners/__init__.py, line 564, in finish_job job_state.job_wrapper.finish( stdout, stderr, exit_code ) File /mnt/galaxy/galaxy-app/lib/galaxy/jobs/__init__.py, line 1107, in finish dataset.datatype.set_meta( dataset, overwrite=False ) # call datatype.set_meta directly for the initial set_meta call during dataset creation File /mnt/galaxy/shed_tools/testtoolshed.g2.bx.psu.edu/repos/iuc/snpeff/1938721334b3/snpeff/lib/galaxy/datatypes/snpeff.py, line 21, in set_meta data_dir = dataset.files_path AttributeError: 'HistoryDatasetAssociation' object has no attribute 'files_path' We fiddled around with the wrapper, eventually replacing 'dataset.files_path' with 'dataset.extra_files_path' in snpeff.py, which fixed the download bug, but then SnpEff subtool itself threw a similar error when I tried to use that database from the history. I chased up a bit more but cannot understand the various posts on files_path vs
Re: [galaxy-dev] Set output dbkey from parameter value
Does removing the ‘param_attribute=“value' attribute help?
On Oct 15, 2014, at 11:23 AM, Nikos Sidiropoulos nikos.sid...@gmail.com wrote:
Hi Daniel,
Thanks for the response.
I've edited the output to:
data format=bedgraph name=bedgraph_slograt
label=${tool.name} on ${on_string}: Smoot Log2ratio (bedGraph)
from_work_dir=output_dir/slograt.bedgraph
filter bedgraph['check'] == 'yes' and slograt['check']
== 'yes' /filter
actions
conditional name=bedgraph.check
when value=yes
action type=metadata name=dbkey
option type=from_param
name=bedgraph.genome param_attribute=value /
/action
/when
/conditional
/actions
/data
Now I'm getting a tool execution error.
Error executing tool: 'unicode' object has no attribute 'value'
I've tried to change the param_attribute to ext, dbkey (ones that
I know that exist) and got a similar error.
Bests,
Nikos
On 15 October 2014 16:58, Daniel Blankenberg d...@bx.psu.edu wrote:
Hi Nikos,
In the very least, you’ll want to make sure that you have a bounding
actions/actions tag set around your actions. It is probably also
advisable to add a set of conditional/whens around the action, since you’re
only setting the dbkey under certain circumstances.
Thanks for using Galaxy,
Dan
On Oct 15, 2014, at 6:37 AM, Nikos Sidiropoulos nikos.sid...@gmail.com
wrote:
Hi all,
I'm trying to set the dbkey of an output file from the value (text) of
a parameter.
The parameter I want to use is genome.
conditional name=bedgraph
param name=check type=select label=Produce BedGraph
output help=Can be displayed directly on UCSC browser. One file per
normalisation method.
option value=no selected=TrueNo/option
option value=yesYes/option
/param
when value=yes
param name=bed_file type=data format=bed
label=Transcripts ins BED format help=12 column BED file containing
trancript definitions. /
param name=genome type=text label=Genome Build
help=E.g. hg19 /
param name=track_name type=text label=Track
Name size=20 value=Track Name /
/when
when value=no /
/conditional
and this is how I've set the output:
data format=bedgraph name=bedgraph_slograt
label=${tool.name} on ${on_string}: Smoot Log2ratio (bedGraph)
from_work_dir=output_dir/slograt.bedgraph
filter bedgraph['check'] == 'yes' and slograt['check']
== 'yes' /filter
action type=metadata name=dbkey
option type=from_param name=bedgraph.genome
param_attribute=value /
/action
/data
When I run the tool the dbkey isn't set to the output file. Does
anyone know a workaround?
Bests,
Nikos
___
Please keep all replies on the list by using reply all
in your mail client. To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
http://lists.bx.psu.edu/
To search Galaxy mailing lists use the unified search at:
http://galaxyproject.org/search/mailinglists/
___
Please keep all replies on the list by using reply all
in your mail client. To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
http://lists.bx.psu.edu/
To search Galaxy mailing lists use the unified search at:
http://galaxyproject.org/search/mailinglists/
Re: [galaxy-dev] Set output dbkey from parameter value
Yes!
Thank you very much Daniel.
On 15 October 2014 18:14, Daniel Blankenberg d...@bx.psu.edu wrote:
Does removing the ‘param_attribute=“value' attribute help?
On Oct 15, 2014, at 11:23 AM, Nikos Sidiropoulos nikos.sid...@gmail.com
wrote:
Hi Daniel,
Thanks for the response.
I've edited the output to:
data format=bedgraph name=bedgraph_slograt
label=${tool.name} on ${on_string}: Smoot Log2ratio (bedGraph)
from_work_dir=output_dir/slograt.bedgraph
filter bedgraph['check'] == 'yes' and slograt['check']
== 'yes' /filter
actions
conditional name=bedgraph.check
when value=yes
action type=metadata name=dbkey
option type=from_param
name=bedgraph.genome param_attribute=value /
/action
/when
/conditional
/actions
/data
Now I'm getting a tool execution error.
Error executing tool: 'unicode' object has no attribute 'value'
I've tried to change the param_attribute to ext, dbkey (ones that
I know that exist) and got a similar error.
Bests,
Nikos
On 15 October 2014 16:58, Daniel Blankenberg d...@bx.psu.edu wrote:
Hi Nikos,
In the very least, you’ll want to make sure that you have a bounding
actions/actions tag set around your actions. It is probably also
advisable to add a set of conditional/whens around the action, since you’re
only setting the dbkey under certain circumstances.
Thanks for using Galaxy,
Dan
On Oct 15, 2014, at 6:37 AM, Nikos Sidiropoulos nikos.sid...@gmail.com
wrote:
Hi all,
I'm trying to set the dbkey of an output file from the value (text) of
a parameter.
The parameter I want to use is genome.
conditional name=bedgraph
param name=check type=select label=Produce BedGraph
output help=Can be displayed directly on UCSC browser. One file per
normalisation method.
option value=no selected=TrueNo/option
option value=yesYes/option
/param
when value=yes
param name=bed_file type=data format=bed
label=Transcripts ins BED format help=12 column BED file containing
trancript definitions. /
param name=genome type=text label=Genome Build
help=E.g. hg19 /
param name=track_name type=text label=Track
Name size=20 value=Track Name /
/when
when value=no /
/conditional
and this is how I've set the output:
data format=bedgraph name=bedgraph_slograt
label=${tool.name} on ${on_string}: Smoot Log2ratio (bedGraph)
from_work_dir=output_dir/slograt.bedgraph
filter bedgraph['check'] == 'yes' and slograt['check']
== 'yes' /filter
action type=metadata name=dbkey
option type=from_param name=bedgraph.genome
param_attribute=value /
/action
/data
When I run the tool the dbkey isn't set to the output file. Does
anyone know a workaround?
Bests,
Nikos
___
Please keep all replies on the list by using reply all
in your mail client. To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
http://lists.bx.psu.edu/
To search Galaxy mailing lists use the unified search at:
http://galaxyproject.org/search/mailinglists/
___
Please keep all replies on the list by using reply all
in your mail client. To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
http://lists.bx.psu.edu/
To search Galaxy mailing lists use the unified search at:
http://galaxyproject.org/search/mailinglists/
Re: [galaxy-dev] snpeff tool for Galaxy extra_files_path
Hey JJ, Opened a pull request to stable with my best guess at the right to proceed and hopefully a best practice recommendation we can all get behind. Do you want to try it out and let me know if it fixes snpeff? (It does fix the velvet datatypes you contributed to Galaxy.) https://bitbucket.org/galaxy/galaxy-central/pull-request/532/fix-for-datatypes-consuming-output-extra/diff Dan, Bjoern - does this make sense - can we move forward with this approach ($input.extra_files_path for inputs and $output.files_path for outputs) as the best practices for how to reference these directories. -John On Wed, Oct 15, 2014 at 11:44 AM, Jim Johnson johns...@umn.edu wrote: I agree with you about the inadvisable use of: input.dataset.*. I'm looking at: lib/galaxy/model/__init__.py class Dataset( object ): ... def __init__( self, id=None, state=None, external_filename=None, extra_files_path=None, file_size=None, purgable=True, uuid=None ): ... self._extra_files_path = extra_files_path ... @property def extra_files_path( self ): return self.object_store.get_filename( self, dir_only=True, extra_dir=self._extra_files_path or dataset_%d_files % self.id ) I'm trying to see when self._extra_files_path gets set. Otherwise, would this return the path relative to the current file location of dataset? On 10/15/14, 9:36 AM, John Chilton wrote: Okay - so this is what broke things: https://bitbucket.org/galaxy/galaxy-central/commits/d781366bc120787e201b73a4dd99b56282169d86 My feeling with the commit was that wrappers and tools should never be explicitly accessing paths explicitly through input.dataset.*. I think this would circumvent options like outputs_to_working_directory and break remote job execution through Pulsar. It also breaks the object store abstraction I think - which is why I made the change for Bjoern I guess. I did not (and this was stupid on my part) realize that datatype code would be running on the remote host and accessing these model properties directly outside the abstractions setup by the wrappers supplied to cheetah code and so they have become out of sync as of that commit. I am thinking somehow changing what the datatype code gets is the right approach and not fixing things by circumvent the wrapper and accessing properties directly on the dataset. Since you will find that doing this breaks things for Bjoern object store and could probably never run on usegalaxy.org say for the same reason. Too many different competing deployment options all being incompatible with each other :(. Will keep thinking about this and respond again. -John On Wed, Oct 15, 2014 at 9:39 AM, John Chilton jmchil...@gmail.com wrote: JJ, Arg this is a mess. I am very sorry about this - I still don't understand extra_files_path versus files_path myself. There are open questions on Peter's blast repo and no one ever followed up on my object store questions about this with Bjoern's issues a couple release cycles ago. We need to get these to work - write documetation explicitly declaring best practices we can all agree on and then write some tests to ensure things don't break in the future. When you say your tools broke recently - can you say for certain which release broke these - the August14, October14, something older? I'll try to do some more research and get back to you. -John On Tue, Oct 14, 2014 at 6:04 AM, Jim Johnson johns...@umn.edu wrote: Andrew, Thanks for investigating this. I changed the subject and sent to the galaxy dev list. I've had a number of tools quit working recently. Particularly tools that inspect the extra_files_path when setting metadata, Defuse, Rsem, SnpEff. I think there was a change in the galaxy framework: The extra_files_path when referenced from an input or output in the cheetah template sections of the tool config xml will be relative to the job working directly rather than the files location. I've just changed a few of my tools on my server yesterday from: param_name.extra_files_path to: param_name.dataset.extra_files_path and they now work again. Dan or John, is that the right way to handle this? Thanks, JJ On 10/13/14, 9:29 PM, Andrew Lonie wrote: Hi Jim. I am probably going about this the wrong way, but I am not clear on how to report tool errors (if in fact this is a tool error!) I've been trialling your snpeff wrapper from the test toolshed and getting a consistent error with the SnpEff Download and SnpEff sub tools (the SnpSift dbNSFP works fine). The problem seems to be with an attribute declaration and manifests during database download as: Traceback (most recent call last): File /mnt/galaxy/galaxy-app/lib/galaxy/jobs/runners/__init__.py, line 564, in finish_job job_state.job_wrapper.finish( stdout, stderr, exit_code ) File /mnt/galaxy/galaxy-app/lib/galaxy/jobs/__init__.py, line 1107, in finish
Re: [galaxy-dev] snpeff tool for Galaxy extra_files_path
Hi John, glad to see this gets some attention! Am 15.10.2014 um 19:05 schrieb John Chilton: Hey JJ, Opened a pull request to stable with my best guess at the right to proceed and hopefully a best practice recommendation we can all get behind. Do you want to try it out and let me know if it fixes snpeff? (It does fix the velvet datatypes you contributed to Galaxy.) https://bitbucket.org/galaxy/galaxy-central/pull-request/532/fix-for-datatypes-consuming-output-extra/diff Dan, Bjoern - does this make sense - can we move forward with this approach ($input.extra_files_path for inputs and $output.files_path for outputs) as the best practices for how to reference these directories. I'm not sure why we need this distinction? Can we not simply choose one for both, inputs and outputs? Otherwise we need to explain it very well, why this is needed and I would vote to rename it to reflect that files_path can be only used by $outputs ... Salve, Bjoern -John On Wed, Oct 15, 2014 at 11:44 AM, Jim Johnson johns...@umn.edu wrote: I agree with you about the inadvisable use of: input.dataset.*. I'm looking at: lib/galaxy/model/__init__.py class Dataset( object ): ... def __init__( self, id=None, state=None, external_filename=None, extra_files_path=None, file_size=None, purgable=True, uuid=None ): ... self._extra_files_path = extra_files_path ... @property def extra_files_path( self ): return self.object_store.get_filename( self, dir_only=True, extra_dir=self._extra_files_path or dataset_%d_files % self.id ) I'm trying to see when self._extra_files_path gets set. Otherwise, would this return the path relative to the current file location of dataset? On 10/15/14, 9:36 AM, John Chilton wrote: Okay - so this is what broke things: https://bitbucket.org/galaxy/galaxy-central/commits/d781366bc120787e201b73a4dd99b56282169d86 My feeling with the commit was that wrappers and tools should never be explicitly accessing paths explicitly through input.dataset.*. I think this would circumvent options like outputs_to_working_directory and break remote job execution through Pulsar. It also breaks the object store abstraction I think - which is why I made the change for Bjoern I guess. I did not (and this was stupid on my part) realize that datatype code would be running on the remote host and accessing these model properties directly outside the abstractions setup by the wrappers supplied to cheetah code and so they have become out of sync as of that commit. I am thinking somehow changing what the datatype code gets is the right approach and not fixing things by circumvent the wrapper and accessing properties directly on the dataset. Since you will find that doing this breaks things for Bjoern object store and could probably never run on usegalaxy.org say for the same reason. Too many different competing deployment options all being incompatible with each other :(. Will keep thinking about this and respond again. -John On Wed, Oct 15, 2014 at 9:39 AM, John Chilton jmchil...@gmail.com wrote: JJ, Arg this is a mess. I am very sorry about this - I still don't understand extra_files_path versus files_path myself. There are open questions on Peter's blast repo and no one ever followed up on my object store questions about this with Bjoern's issues a couple release cycles ago. We need to get these to work - write documetation explicitly declaring best practices we can all agree on and then write some tests to ensure things don't break in the future. When you say your tools broke recently - can you say for certain which release broke these - the August14, October14, something older? I'll try to do some more research and get back to you. -John On Tue, Oct 14, 2014 at 6:04 AM, Jim Johnson johns...@umn.edu wrote: Andrew, Thanks for investigating this. I changed the subject and sent to the galaxy dev list. I've had a number of tools quit working recently. Particularly tools that inspect the extra_files_path when setting metadata, Defuse, Rsem, SnpEff. I think there was a change in the galaxy framework: The extra_files_path when referenced from an input or output in the cheetah template sections of the tool config xml will be relative to the job working directly rather than the files location. I've just changed a few of my tools on my server yesterday from: param_name.extra_files_path to: param_name.dataset.extra_files_path and they now work again. Dan or John, is that the right way to handle this? Thanks, JJ On 10/13/14, 9:29 PM, Andrew Lonie wrote: Hi Jim. I am probably going about this the wrong way, but I am not clear on how to report tool errors (if in fact this is a tool error!) I've been trialling your snpeff wrapper from the test toolshed and getting a consistent error with the SnpEff Download and SnpEff sub tools (the SnpSift dbNSFP works fine). The
Re: [galaxy-dev] snpeff tool for Galaxy extra_files_path
On Wed, Oct 15, 2014 at 1:47 PM, Björn Grüning bjoern.gruen...@gmail.com wrote: Hi John, glad to see this gets some attention! Am 15.10.2014 um 19:05 schrieb John Chilton: Hey JJ, Opened a pull request to stable with my best guess at the right to proceed and hopefully a best practice recommendation we can all get behind. Do you want to try it out and let me know if it fixes snpeff? (It does fix the velvet datatypes you contributed to Galaxy.) https://bitbucket.org/galaxy/galaxy-central/pull-request/532/fix-for-datatypes-consuming-output-extra/diff Dan, Bjoern - does this make sense - can we move forward with this approach ($input.extra_files_path for inputs and $output.files_path for outputs) as the best practices for how to reference these directories. I'm not sure why we need this distinction? Can we not simply choose one for both, inputs and outputs? Otherwise we need to explain it very well, why this is needed and I would vote to rename it to reflect that files_path can be only used by $outputs ... I sympathize with you that this adds complexity - I really do. But if we do anything else we restrict the range of Galaxy versions these tools can target even further - and we still have to maintain backward compatibility on all of this junk anyway which is really weighing down the wrapper and now metadata code as well. If you want input.files_path to work - that is fine - I wouldn't be eager for the change given the complexity it would add to the implementation but I would probably accept a pull request for that. If you want $input.input_files_path and $output.output_files_path to work - I would probably accept pull requests for those as well but I would not be excited. Finally, I don't personally really want to put the time in given my reservations and the benefits would not be so great I don't think because I would think it would be awhile before we could really recommend those as best practices anyway - given the range of Galaxy versions people run. How about we reach an agreement that with a fictitious Tool 2.0 spec (https://trello.com/c/AWVobyv1) where we fix all the problems we will not grant access to $input.dataset directly and we will uniformly only allow $input.files_path and $output.files_path. -John Salve, Bjoern -John On Wed, Oct 15, 2014 at 11:44 AM, Jim Johnson johns...@umn.edu wrote: I agree with you about the inadvisable use of: input.dataset.*. I'm looking at: lib/galaxy/model/__init__.py class Dataset( object ): ... def __init__( self, id=None, state=None, external_filename=None, extra_files_path=None, file_size=None, purgable=True, uuid=None ): ... self._extra_files_path = extra_files_path ... @property def extra_files_path( self ): return self.object_store.get_filename( self, dir_only=True, extra_dir=self._extra_files_path or dataset_%d_files % self.id ) I'm trying to see when self._extra_files_path gets set. Otherwise, would this return the path relative to the current file location of dataset? On 10/15/14, 9:36 AM, John Chilton wrote: Okay - so this is what broke things: https://bitbucket.org/galaxy/galaxy-central/commits/d781366bc120787e201b73a4dd99b56282169d86 My feeling with the commit was that wrappers and tools should never be explicitly accessing paths explicitly through input.dataset.*. I think this would circumvent options like outputs_to_working_directory and break remote job execution through Pulsar. It also breaks the object store abstraction I think - which is why I made the change for Bjoern I guess. I did not (and this was stupid on my part) realize that datatype code would be running on the remote host and accessing these model properties directly outside the abstractions setup by the wrappers supplied to cheetah code and so they have become out of sync as of that commit. I am thinking somehow changing what the datatype code gets is the right approach and not fixing things by circumvent the wrapper and accessing properties directly on the dataset. Since you will find that doing this breaks things for Bjoern object store and could probably never run on usegalaxy.org say for the same reason. Too many different competing deployment options all being incompatible with each other :(. Will keep thinking about this and respond again. -John On Wed, Oct 15, 2014 at 9:39 AM, John Chilton jmchil...@gmail.com wrote: JJ, Arg this is a mess. I am very sorry about this - I still don't understand extra_files_path versus files_path myself. There are open questions on Peter's blast repo and no one ever followed up on my object store questions about this with Bjoern's issues a couple release cycles ago. We need to get these to work - write documetation explicitly declaring best practices we can all agree on and then write some tests to ensure things don't break in the future. When you say your tools broke recently - can you say for
Re: [galaxy-dev] snpeff tool for Galaxy extra_files_path
Looks good, John. I tested with: https://testtoolshed.g2.bx.psu.edu/view/jjohnson/snpsift_dbnsfp_datatypes lib/galaxy/datatypes/converters/tabular_to_dbnsfp.xml reverting from hack: command interpreter=pythontabular_to_dbnsfp.py $input $dbnsfp.dataset.extra_files_path/dbNSFP.gz/command back to: command interpreter=pythontabular_to_dbnsfp.py $input $dbnsfp.files_path/dbNSFP.gz/command On 10/15/14, 12:05 PM, John Chilton wrote: Hey JJ, Opened a pull request to stable with my best guess at the right to proceed and hopefully a best practice recommendation we can all get behind. Do you want to try it out and let me know if it fixes snpeff? (It does fix the velvet datatypes you contributed to Galaxy.) https://bitbucket.org/galaxy/galaxy-central/pull-request/532/fix-for-datatypes-consuming-output-extra/diff Dan, Bjoern - does this make sense - can we move forward with this approach ($input.extra_files_path for inputs and $output.files_path for outputs) as the best practices for how to reference these directories. -John On Wed, Oct 15, 2014 at 11:44 AM, Jim Johnson johns...@umn.edu wrote: I agree with you about the inadvisable use of: input.dataset.*. I'm looking at: lib/galaxy/model/__init__.py class Dataset( object ): ... def __init__( self, id=None, state=None, external_filename=None, extra_files_path=None, file_size=None, purgable=True, uuid=None ): ... self._extra_files_path = extra_files_path ... @property def extra_files_path( self ): return self.object_store.get_filename( self, dir_only=True, extra_dir=self._extra_files_path or dataset_%d_files % self.id ) I'm trying to see when self._extra_files_path gets set. Otherwise, would this return the path relative to the current file location of dataset? On 10/15/14, 9:36 AM, John Chilton wrote: Okay - so this is what broke things: https://bitbucket.org/galaxy/galaxy-central/commits/d781366bc120787e201b73a4dd99b56282169d86 My feeling with the commit was that wrappers and tools should never be explicitly accessing paths explicitly through input.dataset.*. I think this would circumvent options like outputs_to_working_directory and break remote job execution through Pulsar. It also breaks the object store abstraction I think - which is why I made the change for Bjoern I guess. I did not (and this was stupid on my part) realize that datatype code would be running on the remote host and accessing these model properties directly outside the abstractions setup by the wrappers supplied to cheetah code and so they have become out of sync as of that commit. I am thinking somehow changing what the datatype code gets is the right approach and not fixing things by circumvent the wrapper and accessing properties directly on the dataset. Since you will find that doing this breaks things for Bjoern object store and could probably never run on usegalaxy.org say for the same reason. Too many different competing deployment options all being incompatible with each other :(. Will keep thinking about this and respond again. -John On Wed, Oct 15, 2014 at 9:39 AM, John Chilton jmchil...@gmail.com wrote: JJ, Arg this is a mess. I am very sorry about this - I still don't understand extra_files_path versus files_path myself. There are open questions on Peter's blast repo and no one ever followed up on my object store questions about this with Bjoern's issues a couple release cycles ago. We need to get these to work - write documetation explicitly declaring best practices we can all agree on and then write some tests to ensure things don't break in the future. When you say your tools broke recently - can you say for certain which release broke these - the August14, October14, something older? I'll try to do some more research and get back to you. -John On Tue, Oct 14, 2014 at 6:04 AM, Jim Johnson johns...@umn.edu wrote: Andrew, Thanks for investigating this. I changed the subject and sent to the galaxy dev list. I've had a number of tools quit working recently. Particularly tools that inspect the extra_files_path when setting metadata, Defuse, Rsem, SnpEff. I think there was a change in the galaxy framework: The extra_files_path when referenced from an input or output in the cheetah template sections of the tool config xml will be relative to the job working directly rather than the files location. I've just changed a few of my tools on my server yesterday from: param_name.extra_files_path to: param_name.dataset.extra_files_path and they now work again. Dan or John, is that the right way to handle this? Thanks, JJ On 10/13/14, 9:29 PM, Andrew Lonie wrote: Hi Jim. I am probably going about this the wrong way, but I am not clear on how to report tool errors (if in fact this is a tool error!) I've been trialling your snpeff wrapper from the test toolshed and getting a consistent error with the SnpEff Download and SnpEff sub tools (the SnpSift dbNSFP works fine).
[galaxy-dev] snpEff and java issue
Hi Team, I had manually installed the latest version of snpEff based on the recommendation of Pablo and after modifying the XML files I had a working version of snpEff that produced a vcf. However, this morning I reran my workflow and it is failing again, but this time I am getting the following error: Fatal error: Exit code 1 (Error) Exception in thread main java.lang.UnsupportedClassVersionError: ca/mcgill/mcb/pcingola/snpEffect/commandLine/SnpEff : Unsupported major.minor version 51.0 at java.lang.ClassLoader.defineClass1(Native Method) at java.lang.ClassLoader.defineClass(ClassLoader.java:643) at java.security.SecureClassLoader.defineClass(SecureClassLoader.java:142) at java.net.URLClassLoader.defineClass(URLClassLoader.java:277) at java.net.URLClassLoader.access$000(URLClassLoader.java:73) at java.net.URLClassLoader$1.run(URLClassLoader.java:212) at java.security.AccessController.doPrivileged(Native Method) at java.net.URLClassLoader.findClass(URLClassLoader.java:205) at java.lang.ClassLoader.loadClass(ClassLoader.java:323) at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:294) at java.lang.ClassLoader.loadClass(ClassLoader.java:268) Could not find the main class: ca.mcgill.mcb.pcingola.snpEffect.commandLine.SnpEff. Program will exit. Nothing has been changed since I had success. Regards, Iry The information in this email, including attachments, may be confidential and is intended solely for the addressee(s). If you believe you received this email by mistake, please notify the sender by return email as soon as possible. ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
