[galaxy-dev] unable to purge deleted datasets

[Milad Bastami]

Hi all,Using two commands below, I am unable to purge deleted datasets (from a 
local instance of galaxy installed on ubuntu). I have deleted some datasets 
from the history panel and I want to purge only datasets not the history itself.
commands:
1. python scripts/cleanup_datasets/cleanup_datasets.py universe_wsgi.ini -d 0 
-6 -r  2. python scripts/cleanup_datasets/cleanup_datasets.py universe_wsgi.ini 
-d 0 -3 -r results:Purged 0 datasetsFreed disk space:  0
I can't figure out what's wrong. Maybe a privilege problem or something else?? 
Any help would be most appreciated. 
   ___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  http://lists.bx.psu.edu/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/

Re: [galaxy-dev] user account

[Jun Fan]

Hi Martin,

  Thanks for the tip of changing email and username under SQL. I managed 
that (update galaxy_user set email=’h...@haha.com’, username=’haha’ where id = 
NNN) and tried to re-register with the same email. Then I got some error saying 
“Internal Server Error
Galaxy was unable to successfully complete your request
An error occurred.
This may be an intermittent problem due to load or other unpredictable factors, 
reloading the page may address the problem.
The error has been logged to our team.
“
Also the link in the verification email does not do the job. Anyway, I 
used the SQL again to bypass this issue (active=’t’, activation_token=’’)
Problem solved. Many thanks for your help.

Best regards!
Jun

From: emulato...@gmail.com [mailto:emulato...@gmail.com] On Behalf Of Martin 
Cech
Sent: Friday, April 25, 2014 5:05 PM
To: Jun Fan
Cc: galaxy-dev@lists.bx.psu.edu
Subject: Re: [galaxy-dev] user account

Hi Jun,

Galaxy does not really delete users. If you are trying to clear the user, just 
change its email as an admin and register the user again with the same address 
as before.

Martin

On Thu, Apr 24, 2014 at 8:56 PM, Jun Fan 
mailto:j@qmul.ac.uk>> wrote:
Hi Martin,

  Thanks for the hint. After setting track_jobs_in_database to True, the 
status of the new upload is waiting to run, which should be the behavior 
expected.
 Luckily when I modified the setting in universal_wsgi.ini, I noticed that 
there is an option allow_user_deletion = True which enables the user deletion 
in UI. The problem is that even I deleted and purged one email, I still can 
re-register with that email. What is the solution to this? Direct SQL operation?

Best regards!
Jun

From: emulato...@gmail.com 
[mailto:emulato...@gmail.com] On Behalf Of Martin 
Cech
Sent: Thursday, April 24, 2014 9:50 PM

To: Jun Fan
Cc: galaxy-dev@lists.bx.psu.edu
Subject: Re: [galaxy-dev] user account

Hi Jun,

the account activation is meant for multi-processor environments and these have 
to have the following option turned on in the configuration:

track_jobs_in_database = True

Please try to change it.

thank you

Martin



On Thu, Apr 24, 2014 at 4:27 PM, Jun Fan 
mailto:j@qmul.ac.uk>> wrote:
Hi Martin

  I am using Postgresql.

Best regards!
Jun

From: emulato...@gmail.com 
[mailto:emulato...@gmail.com] On Behalf Of Martin 
Cech
Sent: Thursday, April 24, 2014 5:21 PM
To: Jun Fan
Cc: galaxy-dev@lists.bx.psu.edu
Subject: Re: [galaxy-dev] user account

Hi Jun,

what kind of database are you using? This feature might not be supported for 
SQLite (as no production Galaxy should run on it :))

best

Martin
Galaxy Team

On Thu, Apr 24, 2014 at 11:42 AM, Jun Fan 
mailto:j@qmul.ac.uk>> wrote:
Hi everyone

   I followed the instruction at 
https://wiki.galaxyproject.org/Admin/UserAccounts to add email verification.  I 
am using GMAIL SMTP server to send out the verification emails. Everything 
seems fine. If using one of the banned email, an error is there. After 
register, a verification email is successfully sent out and the page does say 
that without verification, no job and upload can be done. However when I tried, 
I CAN upload files and run jobs. The grace period is set to be 0. From the 
comment, I assume that there is no grace period at all which means no user 
activity can be carried out. Could anyone point out what I have done wrongly?

   The by-product of above test is that I need to delete the user I created 
for test. I searched the dev mail list. The most recent one concerning this is 
the one sent by Nate in Nov 2011. I just could not find the py file he 
mentioned, probably that email is too old. I have done a grep search for 
“operations.append” and “item.deleted” and found two files requests_admin.py 
and requests.py under lib/galaxy/webapps/galaxy/controllers/. But they are not 
commented which means that I cannot uncomment them to make “delete user” 
function available in the UI as Nate suggested. Any guidance?

Best regards!
Jun

___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  http://lists.bx.psu.edu/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/



___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  http://lists.bx.psu.edu/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/

Re: [galaxy-dev] Cloudman/Galaxy autoscaling

[Jim McCusker]

Thanks, is the idle time configurable?


On Fri, Apr 25, 2014 at 2:43 PM, Dannon Baker wrote:

> You can review the exact code here( see 'slow_job_turnover') :
> https://bitbucket.org/galaxy/cloudman/src/7b8f04895ad309e0168cb3de66446ae20f3d8b3e/cm/services/autoscale.py?at=default
>
> But, basically, load on any particular node isn't very useful for
> autoscaling in this context because most jobs cannot be split among
> multiple nodes.  What we use is a heristic to determine churn and jobs
> waiting to run.  Generally speaking, if you have jobs waiting to run for a
> bit, your cluster should scale.
>
> -Dannon
>
>
> On Fri, Apr 25, 2014 at 2:37 PM, Jim McCusker 
> wrote:
>
>> What are the exact conditions that will trigger autoscaling in a
>> galaxy/cloudman instance? I'm seeing 100% utilization on the head node but
>> no attempts by cloudman at spinning up new instances. What sort of state is
>> supposed to trigger that?
>>
>> Thanks,
>> Jim
>>
>> ___
>> Please keep all replies on the list by using "reply all"
>> in your mail client.  To manage your subscriptions to this
>> and other Galaxy lists, please use the interface at:
>>   http://lists.bx.psu.edu/
>>
>> To search Galaxy mailing lists use the unified search at:
>>   http://galaxyproject.org/search/mailinglists/
>>
>
>
___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  http://lists.bx.psu.edu/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/

Re: [galaxy-dev] Cloudman/Galaxy autoscaling

[Dannon Baker]

You can review the exact code here( see 'slow_job_turnover') :
https://bitbucket.org/galaxy/cloudman/src/7b8f04895ad309e0168cb3de66446ae20f3d8b3e/cm/services/autoscale.py?at=default

But, basically, load on any particular node isn't very useful for
autoscaling in this context because most jobs cannot be split among
multiple nodes.  What we use is a heristic to determine churn and jobs
waiting to run.  Generally speaking, if you have jobs waiting to run for a
bit, your cluster should scale.

-Dannon


On Fri, Apr 25, 2014 at 2:37 PM, Jim McCusker wrote:

> What are the exact conditions that will trigger autoscaling in a
> galaxy/cloudman instance? I'm seeing 100% utilization on the head node but
> no attempts by cloudman at spinning up new instances. What sort of state is
> supposed to trigger that?
>
> Thanks,
> Jim
>
> ___
> Please keep all replies on the list by using "reply all"
> in your mail client.  To manage your subscriptions to this
> and other Galaxy lists, please use the interface at:
>   http://lists.bx.psu.edu/
>
> To search Galaxy mailing lists use the unified search at:
>   http://galaxyproject.org/search/mailinglists/
>
___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  http://lists.bx.psu.edu/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/

[galaxy-dev] Cloudman/Galaxy autoscaling

[Jim McCusker]

What are the exact conditions that will trigger autoscaling in a
galaxy/cloudman instance? I'm seeing 100% utilization on the head node but
no attempts by cloudman at spinning up new instances. What sort of state is
supposed to trigger that?

Thanks,
Jim
___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  http://lists.bx.psu.edu/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/

Re: [galaxy-dev] Limits for enumerate for multiple input files

[John Chilton]

Don't have a ton to add to Peter's suggestion I just wanted chime in
that there is definitely not some inherit Galaxy limit on the backend
for the number selections allow in a multiple input widget. I just
tried a test with 4 times this number of inputs and it worked fine.
This could well be reaching some sort of maximum command line length -
but 20 some files seems a very low limit - in the past when I have hit
such limits it has been on hundreds or thousands of files - not
dozens.

It is difficult to work with a large number of inputs because select2
takes over in the UI and provides a very frustrating data input
experience for these inputs (see conversation here
http://dev.list.galaxyproject.org/HTML-form-of-select-parameter-tt4663142.html).
It may be worth applying this patch
(https://bitbucket.org/galaxy/galaxy-central/pull-request/379/disable-select2-for-all-multi-select/diff)
suggested by Jeremy in that thread to simplify the tool UX and try
again.

-John

On Tue, Apr 22, 2014 at 11:23 AM, Peter Cock  wrote:
> On Tue, Apr 22, 2014 at 4:34 PM, Ulf Schaefer  wrote:
>> Hi Peter
>>
>> I removed the unnecessary code.
>>
>> If I run the tool with just a couple of inputs I see entries in the log
>> files either from galaxy.jobs.runners.drmaa or from
>> galaxy.jobs.runners.local that the job is being dispatched as normal.
>>
>> Unfortunately there is no sign of the job in the log files when using
>> more input files.
>>
>> The command line that is supposed to be run is:
>>
>> bash home/galaxy/galaxy-dist/tools/vcf_processing/vcf_to_fasta.sh
>> /galaxy/database/files/042/dataset_42275.dat 40 10 50 0 40 0.9 20
>> /galaxy/database/files/041/dataset_41720.dat,
>> /galaxy/database/files/041/dataset_41980.dat,
>>
>> the first dat file being the output and the ones at the end being a
>> comma separated list of the input files. On the command line this
>> command works with much longer input files lists.
>
> I wouldn't bother with the commas - that is just wasting characters
> and eating into the maximum command line string length.
>
>> Any ideas?
>
> Check the limit with "xargs --show-limits" or "getconf ARG_MAX",
> our CentOS server reports:
>
> $ getconf ARG_MAX
> 2621440
>
>> Or is there a better practice to pass a large number of input
>> files to a bash script?
>>
>> Thanks
>> Ulf
>
> If there is any chance of your constructed command line string
> exceeding the system limit, I would construct an input file
> containing the filenames (e.g. one per line). That might be
> a practical solution anyway.
>
> For the file bases approach, I would used the Galaxy 
> tag. Some of the tools bundled with Galaxy also use this (find
> them with grep), or for example one of mine:
>
> https://github.com/peterjc/pico_galaxy/blob/master/tools/mira4/mira4_de_novo.xml
>
> Regards,
>
> Peter
> ___
> Please keep all replies on the list by using "reply all"
> in your mail client.  To manage your subscriptions to this
> and other Galaxy lists, please use the interface at:
>   http://lists.bx.psu.edu/
>
> To search Galaxy mailing lists use the unified search at:
>   http://galaxyproject.org/search/mailinglists/
___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  http://lists.bx.psu.edu/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/

Re: [galaxy-dev] "Intersect multiple sorted BED files" / tool-data/shared/ucsc/chrom/genome_test.len) could not be opened

[Jennifer Jackson]

Sarah,

To my knowledge, Trackster has upgraded quite a bit since last Nov. Try 
upgrading and see if that helps. Be sure to see the News Brief in the 
link I sent for the full instructions linked in the first top section 
titled "Upgrades". Resetting metadata and other items may require 
upgrades along with the core dist repository. This was required for the 
Nov upgrade and can be done at any time. Will help to ensure you have 
fully up-to-date instance now, a good idea for troubleshooting.


Reach all News Briefs here:
https://wiki.galaxyproject.org/DevNewsBriefs

It is difficult to troubleshoot an older distribution potentially mixed 
with more recent changesets (these often occur in group, or have other 
dependencies). Starting with a stable distribution is best, especially 
when problems are encountered.


Our development team will send more if there additional advice. Please 
let us know how this works out.


Jen
Galaxy team

On 4/25/14 6:33 AM, Sarah Maman wrote:

Thanks Jennifer,

My problem is not solved yet.
The Galaxy version is not so old - November 2013 (* please see below)
len_file_path is already uncommented in universe.
Chromosome and genome are availables in loc files and 
tool-data/shared/ucsc/chrom also.


Best,
Sarah



Galaxy Version (*) :
galaxy-dev@genotoul /usr/local/bioinfo/src/galaxy-dev/galaxy-dist $ hg 
heads

changeset:   11219:5c789ab4144a
branch:  stable
tag: tip
user:Nate Coraor 
date:Mon Nov 04 15:04:42 2013 -0500
summary: Added tag release_2013.11.04 for changeset 26f58e05aa10

changeset:   11216:c458a0fe1ba8
parent:  11213:6d633418ecfa
parent:  11215:f79149dd3d35
user:Nate Coraor 
date:Mon Nov 04 14:56:57 2013 -0500
summary: Merge security fix for filtering tools from 
stable/next-stable.


changeset:   11168:654dfe820960
branch:  search
parent:  11166:776ca4a63a0b
user:Dannon Baker 
date:Fri Oct 25 15:32:18 2013 -0400
summary: Close search branch.

changeset:   11139:7286338fd77e
branch:  extract_genomic_dna_tool_enhancements
parent:  11137:7b014440dda1
user:John Chilton 
date:Fri Oct 25 00:09:25 2013 -0400
summary: Close branch extract_genomic_dna_tool_enhancements.

changeset:   11129:471f23116c5e
branch:  tool_api
parent:  11127:edab395a78b1
user:John Chilton 
date:Thu Oct 24 21:08:11 2013 -0500
summary: Close branch tool_api

changeset:   11126:e19d39296919
branch:  search
parent:  11124:35a847f0f33d
user:Dannon Baker 
date:Thu Oct 24 18:12:50 2013 -0400
summary: Re-close search feature branch.

changeset:   11087:28d3ef3d5ed5
branch:  search
parent:  11085:f82bbdc5c22b
user:John Chilton 
date:Tue Oct 22 01:47:05 2013 -0500
summary: Close branch search

changeset:   11017:c8bbd1d6c18c
branch:  api_copy_history
parent:  11015:138901353da9
user:Carl Eberhard 
date:Fri Oct 18 14:53:26 2013 -0400
summary: Close branch: api_copy_history

changeset:   11002:acf1a0062ea8
branch:  custom-toolbox2
parent:  10997:d816d6602e18
user:Dannon Baker 
date:Thu Oct 17 15:32:40 2013 -0400
summary: Close feature branch for toolbox filtering 2

changeset:   10968:eeebcddfa8f4
branch: abrenner/histogram2xml-python-module-rpy-requires-1379010965302
parent:  10964:913f3ab84629
user:John Chilton 
date:Thu Oct 17 04:06:42 2013 +
summary: Close branch 
abrenner/histogram2xml-python-module-rpy-requires-1379010965302.


changeset:   10935:8b7e05aae54e
branch:  action_tpye_autoconf
parent:  10933:5a26cbd98609
user:Dave B. 
date:Tue Oct 15 15:21:52 2013 -0400
summary: Close branch action_tpye_autoconf

changeset:   10923:914cec313725
branch:  action_tpye_make_install
parent:  10921:be307f53645c
user:Dave Bouvier 
date:Tue Oct 15 11:32:42 2013 -0400
summary: Close branch action_tpye_make_install.

changeset:   10919:e3e016f0f6f3
branch:  R_environment
parent:  10917:e19c7819ea3c
user:Dave B. 
date:Tue Oct 15 11:24:21 2013 -0400
summary: Close branch R_environment

changeset:   10913:a2b2de536a47
branch:  action_type_misc
parent:  10911:223f968a3b3c
user:Dave B. 
date:Tue Oct 15 11:15:57 2013 -0400
summary: Close branch action_type_misc

changeset:   10612:e3e7b034cb0b
branch:  tool_api_submit
parent:  10608:abb889f8c5b7
user:Carl Eberhard 
date:Mon Sep 16 15:45:04 2013 -0400
summary: Close branch tool_api_submit

changeset:   10564:684ef8edd1d4
branch:  dataset-cleanup
parent:  10530:e4734c99812a
user:jeremy goecks 
date:Wed Sep 11 12:24:00 2013 -0400
summary: Close branch 10525:e4734c99812a

changeset:   10401:53c6769dcff4
branch:  cleanup_datasets_unicode_fix
parent:  10399:b245b46ec4c0
user:Dannon Baker 
date:Thu Aug 15 11:20:

Re: [galaxy-dev] workflow_execute_parameters.py

[Dannon Baker]

Olivia,

Is the parameter set to 'set at runtime' when you view the workflow in the
editor?  Could you send me the workflow json (just the
http://127.0.0.1:8080/api/workflows// bit)?

-Dannon


On Fri, Apr 25, 2014 at 5:26 AM, Olivia Doppelt-Azeroual <
olivia.dopp...@pasteur.fr> wrote:

>  Dear Dannon,
>
> Thank you for your answer, it works better now that I know the steps ID
> for my workflow.
>
> I have another question though.
>
> The second step of my WF uses a parameter input named "prefix". I really
> need to specify it when I launch the WF using the API.
>
> When i display the WF using the url you gave me, the parameters prefix, is
> not there and when I use this in my command line, the specified Value is
> not read by Galaxy.
>
> python workflow_execute_parameters.py 43359c833b8977b01d6237d9b839dc27
> http://127.0.0.1:8080/api/workflows f2db41e1fa331b3e a799d38679e985db
> '10=hda=52ea4fd71e9f7c3f'  'param=export_sbw=prefix=test'
>
> Could you help me on that ? How do I give a value to a parameter if it
> does not appear in the WF description ??
>
> Thanks in advance,
> --
> Olivia
>
>
> Le 23/04/14 18:40, Dannon Baker a écrit :
>
> Hi Olivia!
>
>  It's probably talking about the "workflowstep" object with identifier 8.
>  If you request the description of this workflow via the API (
> http://yourgalaxyinstance.org/api/workflows/"),
> you'll be given the exact step identifiers to use.  In other words, unless
> this is the first workflow ever created on a brand new galaxy instance,
> your step ids will not be 0 and 1.
>
>  -Dannon
>
>
> On Wed, Apr 23, 2014 at 12:30 PM, Olivia Doppelt-Azeroual <
> olivia.dopp...@pasteur.fr> wrote:
>
>> Dear developpers,
>>
>> I'm trying to launch a small workflow using the api script
>> workflow_execute_parameters.py
>>
>> I have 2 steps: the input data step and a second step which needs a
>> parameter called prefix.
>>
>>
>> The workflow is attached to this email.
>>
>> I get an error on the parameters inputs which asks me to specify
>> information about an 8th step (which does not exist)
>>
>>
>> Is there a test case available to use this script ?
>>
>> Please keep me informed on that.
>>
>> Have a nice day,
>>
>> --
>> Olivia Doppelt-Azeroual, PhD
>> Tel: 92 15
>> CIB - Institut Pasteur
>>
>>
>> ___
>> Please keep all replies on the list by using "reply all"
>> in your mail client.  To manage your subscriptions to this
>> and other Galaxy lists, please use the interface at:
>>   http://lists.bx.psu.edu/
>>
>> To search Galaxy mailing lists use the unified search at:
>>   http://galaxyproject.org/search/mailinglists/
>>
>
>
>
> --
> Olivia Doppelt-Azeroual, PhD
> Tel: 92 15
> CIB - Institut Pasteur
>
>
___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  http://lists.bx.psu.edu/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/

Re: [galaxy-dev] "Intersect multiple sorted BED files" / tool-data/shared/ucsc/chrom/genome_test.len) could not be opened

[Jennifer Jackson]

Sarah,

To my knowledge, Trackster has upgraded quite a bit since last Nov. Try 
upgrading and see if that helps. Be sure to see the News Brief in the 
link I sent for the full instructions linked in the first top section 
titled "Upgrades". Resetting metadata and other items may require 
upgrades along with the core dist repository. This was required for the 
Nov upgrade and can be done at any time. Will help to ensure you have 
fully up-to-date instance now, a good idea for troubleshooting.


Reach all News Briefs here:
https://wiki.galaxyproject.org/DevNewsBriefs

It is difficult to troubleshoot an older distribution potentially mixed 
with more recent changesets (these often occur in group, or have other 
dependencies). Starting with a stable distribution is best, especially 
when problems are encountered.


Our development team will send more if there additional advice. Please 
let us know how this works out.


Jen
Galaxy team

On 4/25/14 6:33 AM, Sarah Maman wrote:

Thanks Jennifer,

My problem is not solved yet.
The Galaxy version is not so old - November 2013 (* please see below)
len_file_path is already uncommented in universe.
Chromosome and genome are availables in loc files and 
tool-data/shared/ucsc/chrom also.


Best,
Sarah



Galaxy Version (*) :
galaxy-dev@genotoul /usr/local/bioinfo/src/galaxy-dev/galaxy-dist $ hg 
heads

changeset:   11219:5c789ab4144a
branch:  stable
tag: tip
user:Nate Coraor 
date:Mon Nov 04 15:04:42 2013 -0500
summary: Added tag release_2013.11.04 for changeset 26f58e05aa10

changeset:   11216:c458a0fe1ba8
parent:  11213:6d633418ecfa
parent:  11215:f79149dd3d35
user:Nate Coraor 
date:Mon Nov 04 14:56:57 2013 -0500
summary: Merge security fix for filtering tools from 
stable/next-stable.


changeset:   11168:654dfe820960
branch:  search
parent:  11166:776ca4a63a0b
user:Dannon Baker 
date:Fri Oct 25 15:32:18 2013 -0400
summary: Close search branch.

changeset:   11139:7286338fd77e
branch:  extract_genomic_dna_tool_enhancements
parent:  11137:7b014440dda1
user:John Chilton 
date:Fri Oct 25 00:09:25 2013 -0400
summary: Close branch extract_genomic_dna_tool_enhancements.

changeset:   11129:471f23116c5e
branch:  tool_api
parent:  11127:edab395a78b1
user:John Chilton 
date:Thu Oct 24 21:08:11 2013 -0500
summary: Close branch tool_api

changeset:   11126:e19d39296919
branch:  search
parent:  11124:35a847f0f33d
user:Dannon Baker 
date:Thu Oct 24 18:12:50 2013 -0400
summary: Re-close search feature branch.

changeset:   11087:28d3ef3d5ed5
branch:  search
parent:  11085:f82bbdc5c22b
user:John Chilton 
date:Tue Oct 22 01:47:05 2013 -0500
summary: Close branch search

changeset:   11017:c8bbd1d6c18c
branch:  api_copy_history
parent:  11015:138901353da9
user:Carl Eberhard 
date:Fri Oct 18 14:53:26 2013 -0400
summary: Close branch: api_copy_history

changeset:   11002:acf1a0062ea8
branch:  custom-toolbox2
parent:  10997:d816d6602e18
user:Dannon Baker 
date:Thu Oct 17 15:32:40 2013 -0400
summary: Close feature branch for toolbox filtering 2

changeset:   10968:eeebcddfa8f4
branch: abrenner/histogram2xml-python-module-rpy-requires-1379010965302
parent:  10964:913f3ab84629
user:John Chilton 
date:Thu Oct 17 04:06:42 2013 +
summary: Close branch 
abrenner/histogram2xml-python-module-rpy-requires-1379010965302.


changeset:   10935:8b7e05aae54e
branch:  action_tpye_autoconf
parent:  10933:5a26cbd98609
user:Dave B. 
date:Tue Oct 15 15:21:52 2013 -0400
summary: Close branch action_tpye_autoconf

changeset:   10923:914cec313725
branch:  action_tpye_make_install
parent:  10921:be307f53645c
user:Dave Bouvier 
date:Tue Oct 15 11:32:42 2013 -0400
summary: Close branch action_tpye_make_install.

changeset:   10919:e3e016f0f6f3
branch:  R_environment
parent:  10917:e19c7819ea3c
user:Dave B. 
date:Tue Oct 15 11:24:21 2013 -0400
summary: Close branch R_environment

changeset:   10913:a2b2de536a47
branch:  action_type_misc
parent:  10911:223f968a3b3c
user:Dave B. 
date:Tue Oct 15 11:15:57 2013 -0400
summary: Close branch action_type_misc

changeset:   10612:e3e7b034cb0b
branch:  tool_api_submit
parent:  10608:abb889f8c5b7
user:Carl Eberhard 
date:Mon Sep 16 15:45:04 2013 -0400
summary: Close branch tool_api_submit

changeset:   10564:684ef8edd1d4
branch:  dataset-cleanup
parent:  10530:e4734c99812a
user:jeremy goecks 
date:Wed Sep 11 12:24:00 2013 -0400
summary: Close branch 10525:e4734c99812a

changeset:   10401:53c6769dcff4
branch:  cleanup_datasets_unicode_fix
parent:  10399:b245b46ec4c0
user:Dannon Baker 
date:Thu Aug 15 11:20:

Re: [galaxy-dev] DRMAA configuring issue

[John Chilton]

I feel like someone should respond to this but I must admin I don't
have a lot of ideas.

I assume you are able to use qsub to submit jobs from the Galaxy
server? This is worth verifying that before anything else. If that
doesn't work - the system configuration needs to be modified.

I think there are a couple different implementations of DRMAA for PBS:

http://apps.man.poznan.pl/trac/pbs-drmaa (I think this is recommend one).
http://sourceforge.net/projects/pbspro-drmaa/

It might be worth trying to compile the latest and great of one or
both and target both.

Galaxy also has a PBS runner that many people use for communicating
with Torque. I think the DRMAA runner should work - but this is a
fallback option as well just to get going.

-John
___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  http://lists.bx.psu.edu/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/

Re: [galaxy-dev] problem with output file

[John Chilton]

Can you attach the wrapper that produces the error? Does the dataset
have any standard error or standard output messages at the end of the
run? Also the Galaxy logs surrounding when the job runs might help
debug the problem.

On Fri, Apr 18, 2014 at 8:22 AM, Clement Lionnet
 wrote:
> Hi,
>
> I would like to devellop a wrapper with the script EMIRGE. I have a problem 
> with output file because the script creates one folder which containt other 
> folder and i would like to save all folder and get back two file containt in 
> the last folder. My wrapper create folder but at the end my dataset show an 
> error and I don't now why.
>
> ___
> Please keep all replies on the list by using "reply all"
> in your mail client.  To manage your subscriptions to this
> and other Galaxy lists, please use the interface at:
>   http://lists.bx.psu.edu/
>
> To search Galaxy mailing lists use the unified search at:
>   http://galaxyproject.org/search/mailinglists/

___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  http://lists.bx.psu.edu/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/

Re: [galaxy-dev] Galaxy - Cloudman

[Alessia Deglincerti]

Hey Dannon,

I am using Galaxy (Cloudman) to analyze RNA seq data. I have a bunch of 
datasets that I ran TopHat on. I am now trying to run Cufflinks etc. I used the 
same data and workflow yesterday to run Cufflinks etc and that worked fine. I 
used my own reference annotation (Homo sapiens hg38) that I imported from UCSC 
but I wanted to compare to some older data I have and so now I  trying to run 
it on hg17. I also have some other datasets in the queue. All my jobs are gray 
and have been waiting to ran for the past 24+ hours. I am seeing the error as I 
am waiting for my jobs to run, I get a notification at the top of my 
queue/history saying that there was a problem getting updates and to contact a 
site administrator if the problem persists (I've seen the notification a few 
times) and when I click the "details" link in the notification it prompts me 
with the details pasted below. 

I am using all of the default options for my jobs and the same workflow has 
been working fine for me in the past on the same datasets. 

Thanks for any help you can give me!

Alessia 



> On Apr 24, 2014, at 10:24 PM, Dannon Baker  wrote:
> 
> Hey Alessia,
> 
> I'd love to help, but I need a little more context here.  Where are you 
> seeing this error, what exactly are you doing when it appears, and can you 
> tell me a little bit more about the Galaxy configuration you're working with?
> 
> -Dannon
> 
> 
>> On Thu, Apr 24, 2014 at 8:37 PM, Jennifer Jackson  wrote:
>> Posting to the galaxy-...@bx.psu.edu mailing list.
>> https://wiki.galaxyproject.org/MailingLists
>> 
>>> My instance of Galaxy seems to be stuck (same place since last night) and I 
>>> have received the error message several times now.  
>>> 
>>> Pasting the error message below:
>>> 
>>> Details
>>> user
>>> username
>>> adeglincer
>>> quota_percent
>>> 68
>>> total_disk_usage
>>> 184947273683
>>> nice_total_disk_usage
>>> 172.2 GB
>>> email
>>> xxx
>>> is_admin
>>> false
>>> tags_used
>>> 
>>> model_class
>>> User
>>> id
>>> xxx
>>> source
>>> HDACollection(xxx)
>>> xhr
>>> readyState
>>> 4
>>> responseText
>>> {"err_msg": "Uncaught exception in exposed API method:", "err_code": 0}
>>> responseJSON
>>> err_msg
>>> Uncaught exception in exposed API method:
>>> err_code
>>> 0
>>> status
>>> 500
>>> statusText
>>> Internal Server Error
>>> responseHeaders
>>> Date
>>> Thu, 24 Apr 2014 20:51:39 GMT
>>> cache-control
>>> max-age=0,no-cache,no-store
>>> Server
>>> nginx/1.4.7
>>> Connection
>>> keep-alive
>>> Transfer-Encoding
>>> chunked
>>> Content-Type
>>> application/json
>>> options
>>> data
>>> 
>>> parse
>>> true
>>> emulateHTTP
>>> false
>>> emulateJSON
>>> false
>> 
>> ___
>> Please keep all replies on the list by using "reply all"
>> in your mail client.  To manage your subscriptions to this
>> and other Galaxy lists, please use the interface at:
>>   http://lists.bx.psu.edu/
>> 
>> To search Galaxy mailing lists use the unified search at:
>>   http://galaxyproject.org/search/mailinglists/
> 
___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  http://lists.bx.psu.edu/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/

[galaxy-dev] Character encoding on the command line

[Jim McCusker]

When I pass a parameter on the command line certain characters (like @) are
transformed to X+hexadecimal. Is this some sort of sanitization step? Would
using a config file (
https://wiki.galaxyproject.org/Admin/Tools/ToolConfigSyntax#A.3Cconfigfile.3E_tag_set)
avoid that sort of mess?

I'm not sure I've seen this documented (I could be wrong though).

Thanks,
Jim
___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  http://lists.bx.psu.edu/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/

Re: [galaxy-dev] Galaxy/Cloudman Compute Nodes User

[Jim McCusker]

Actually, both of these help. I'm putting a boto.conf file into
/mnt/galaxy/galaxy-app and setting the BOTO_PATH environment variable from
the python script that needs it, so it will work in dev (non-clusterman
setup) and production.

Jim


On Fri, Apr 25, 2014 at 11:20 AM, Dannon Baker wrote:

> Ahh, sorry about that, I misread the bit about it actually needing to be
> in the user's directory.  You could use a worker_post_start_script to
> configure things like this (creating/copying .boto at boot from the shared
> space in the script).  See
> https://wiki.galaxyproject.org/CloudMan/UserData for how to specify that
> in your UserData.
>
> -Dannon
>
>
>
> On Fri, Apr 25, 2014 at 11:13 AM, Dannon Baker wrote:
>
>> Hi Jim,
>>
>> /mnt/galaxy is shared via NFS with all cluster workers -- this (or a
>> subdirectory) should work for you.
>>
>> -Dannon
>>
>>
>> On Fri, Apr 25, 2014 at 10:56 AM, Jim McCusker <
>> jmccus...@5amsolutions.com> wrote:
>>
>>> Hi all,
>>>
>>> I need to add a .boto file to the user that galaxy jobs get exec'ed as
>>> on cloudman nodes. I thought it would copy over what the galaxy server has
>>> in its home directory, but that doesn't seem to happen. How can I get
>>> things like my .boto file to migrate over to new cloudman nodes when they
>>> are spun up?
>>>
>>> Thanks,
>>> Jim
>>>
>>> ___
>>> Please keep all replies on the list by using "reply all"
>>> in your mail client.  To manage your subscriptions to this
>>> and other Galaxy lists, please use the interface at:
>>>   http://lists.bx.psu.edu/
>>>
>>> To search Galaxy mailing lists use the unified search at:
>>>   http://galaxyproject.org/search/mailinglists/
>>>
>>
>>
>
___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  http://lists.bx.psu.edu/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/

Re: [galaxy-dev] user account

[Martin Čech]

Hi Jun,

Galaxy does not really delete users. If you are trying to clear the user,
just change its email as an admin and register the user again with the same
address as before.

Martin


On Thu, Apr 24, 2014 at 8:56 PM, Jun Fan  wrote:

>  Hi Martin,
>
>
>
>   Thanks for the hint. After setting track_jobs_in_database to True,
> the status of the new upload is waiting to run, which should be the
> behavior expected.
>
>  Luckily when I modified the setting in universal_wsgi.ini, I noticed
> that there is an option allow_user_deletion = True which enables the user
> deletion in UI. The problem is that even I deleted and purged one email, I
> still can re-register with that email. What is the solution to this? Direct
> SQL operation?
>
>
>
> Best regards!
>
> Jun
>
>
>
> *From:* emulato...@gmail.com [mailto:emulato...@gmail.com] *On Behalf Of 
> *Martin
> Cech
> *Sent:* Thursday, April 24, 2014 9:50 PM
>
> *To:* Jun Fan
> *Cc:* galaxy-dev@lists.bx.psu.edu
> *Subject:* Re: [galaxy-dev] user account
>
>
>
> Hi Jun,
>
>
>
> the account activation is meant for multi-processor environments and these
> have to have the following option turned on in the configuration:
>
>
>
> track_jobs_in_database = True
>
>
>
> Please try to change it.
>
>
>
> thank you
>
>
>
> Martin
>
>
>
>
>
>
>
> On Thu, Apr 24, 2014 at 4:27 PM, Jun Fan  wrote:
>
> Hi Martin
>
>
>
>   I am using Postgresql.
>
>
>
> Best regards!
>
> Jun
>
>
>
> *From:* emulato...@gmail.com [mailto:emulato...@gmail.com] *On Behalf Of 
> *Martin
> Cech
> *Sent:* Thursday, April 24, 2014 5:21 PM
> *To:* Jun Fan
> *Cc:* galaxy-dev@lists.bx.psu.edu
> *Subject:* Re: [galaxy-dev] user account
>
>
>
> Hi Jun,
>
>
>
> what kind of database are you using? This feature might not be supported
> for SQLite (as no production Galaxy should run on it :))
>
>
>
> best
>
>
>
> Martin
>
> Galaxy Team
>
>
>
> On Thu, Apr 24, 2014 at 11:42 AM, Jun Fan  wrote:
>
> Hi everyone
>
>
>
>I followed the instruction at
> https://wiki.galaxyproject.org/Admin/UserAccounts to add email
> verification.  I am using GMAIL SMTP server to send out the verification
> emails. Everything seems fine. If using one of the banned email, an error
> is there. After register, a verification email is successfully sent out and
> the page does say that without verification, no job and upload can be done.
> However when I tried, I CAN upload files and run jobs. The grace period is
> set to be 0. From the comment, I assume that there is no grace period at
> all which means no user activity can be carried out. Could anyone point out
> what I have done wrongly?
>
>
>
>The by-product of above test is that I need to delete the user I
> created for test. I searched the dev mail list. The most recent one
> concerning this is the one sent by Nate in Nov 2011. I just could not find
> the py file he mentioned, probably that email is too old. I have done a
> grep search for “operations.append” and “item.deleted” and found two files 
> requests_admin.py
> and requests.py under lib/galaxy/webapps/galaxy/controllers/. But they are
> not commented which means that I cannot uncomment them to make “delete
> user” function available in the UI as Nate suggested. Any guidance?
>
>
>
> Best regards!
>
> Jun
>
>
> ___
> Please keep all replies on the list by using "reply all"
> in your mail client.  To manage your subscriptions to this
> and other Galaxy lists, please use the interface at:
>   http://lists.bx.psu.edu/
>
> To search Galaxy mailing lists use the unified search at:
>   http://galaxyproject.org/search/mailinglists/
>
>
>
>
>
___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  http://lists.bx.psu.edu/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/

Re: [galaxy-dev] Galaxy/Cloudman Compute Nodes User

[Dannon Baker]

Ahh, sorry about that, I misread the bit about it actually needing to be in
the user's directory.  You could use a worker_post_start_script to
configure things like this (creating/copying .boto at boot from the shared
space in the script).  See
https://wiki.galaxyproject.org/CloudMan/UserDatafor how to specify
that in your UserData.

-Dannon



On Fri, Apr 25, 2014 at 11:13 AM, Dannon Baker wrote:

> Hi Jim,
>
> /mnt/galaxy is shared via NFS with all cluster workers -- this (or a
> subdirectory) should work for you.
>
> -Dannon
>
>
> On Fri, Apr 25, 2014 at 10:56 AM, Jim McCusker  > wrote:
>
>> Hi all,
>>
>> I need to add a .boto file to the user that galaxy jobs get exec'ed as on
>> cloudman nodes. I thought it would copy over what the galaxy server has in
>> its home directory, but that doesn't seem to happen. How can I get things
>> like my .boto file to migrate over to new cloudman nodes when they are spun
>> up?
>>
>> Thanks,
>> Jim
>>
>> ___
>> Please keep all replies on the list by using "reply all"
>> in your mail client.  To manage your subscriptions to this
>> and other Galaxy lists, please use the interface at:
>>   http://lists.bx.psu.edu/
>>
>> To search Galaxy mailing lists use the unified search at:
>>   http://galaxyproject.org/search/mailinglists/
>>
>
>
___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  http://lists.bx.psu.edu/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/

Re: [galaxy-dev] Galaxy/Cloudman Compute Nodes User

[Dannon Baker]

Hi Jim,

/mnt/galaxy is shared via NFS with all cluster workers -- this (or a
subdirectory) should work for you.

-Dannon


On Fri, Apr 25, 2014 at 10:56 AM, Jim McCusker
wrote:

> Hi all,
>
> I need to add a .boto file to the user that galaxy jobs get exec'ed as on
> cloudman nodes. I thought it would copy over what the galaxy server has in
> its home directory, but that doesn't seem to happen. How can I get things
> like my .boto file to migrate over to new cloudman nodes when they are spun
> up?
>
> Thanks,
> Jim
>
> ___
> Please keep all replies on the list by using "reply all"
> in your mail client.  To manage your subscriptions to this
> and other Galaxy lists, please use the interface at:
>   http://lists.bx.psu.edu/
>
> To search Galaxy mailing lists use the unified search at:
>   http://galaxyproject.org/search/mailinglists/
>
___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  http://lists.bx.psu.edu/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/

[galaxy-dev] Galaxy/Cloudman Compute Nodes User

[Jim McCusker]

Hi all,

I need to add a .boto file to the user that galaxy jobs get exec'ed as on
cloudman nodes. I thought it would copy over what the galaxy server has in
its home directory, but that doesn't seem to happen. How can I get things
like my .boto file to migrate over to new cloudman nodes when they are spun
up?

Thanks,
Jim
___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  http://lists.bx.psu.edu/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/

Re: [galaxy-dev] A proposed modules extension for toolshed wrappers

[John Chilton]

Great - I have added the option to the pull request
https://bitbucket.org/jmchilton/galaxy-central-fork-1/commits/d72967b2b7b25ba02584880526c7b28cab6970e7
as well as an option to supply a command instead of a file so you can
specify the module load directly in job_conf.xml without having to
create a file. Barring objections, I will merge this pull request
later today or early next week and it should be available in the June
release.

All of that said - I agree with Bjoern approach completely and would
go one step further - if working with the packager to modify the tool
to add a package tag that you can create a standard env.sh file for
that would be the 'recommend' solution :).

-John

On Fri, Apr 25, 2014 at 9:06 AM, Björn Grüning
 wrote:
> Hi Ida,
>
> now I got it.
> I may have an idea, not tested, but you could try the following.
>
> - Add a file under $galaxy_tools/R_env/3.1.0/env.sh
> - Create a symlink from $galaxy_tools/R_env/default/ to
> $galaxy_tools/R_env/3.1.0/
> - change your requirement tag to
> R_env
>
> Now you can put in everything you want have in the env.sh file. That file
> will be sourced as soon as your tool will be executed. For example you can
> load your module.
>
> Hope that works!
> Thanks,
> Bjoern
>
>
>> Dear  Björn,
>>
>> maybe I could change the tool, which is not mine, and which I don't want
>> to maintain to use a specific R-version that is already available on our
>> cluster and
>> which I can put into my path with "module load R/3.1.0-devel" (
>> http://modules.sourceforge.net/)
>>
>> there even is a requirements in the wrapper which is not fulfilled (these
>> versions are not available at the moment) and still it installed without
>> problems.
>>
>>  edgeR
>>  limma
>>
>>
>> This requirements tags looks also rather inflexible to me. With an
>> additional level of user configurable indirection it would be possible to
>> make the tools
>> fit to different infrastructures without having to use binaries provided
>> by somebody else, taking up space for just one tool etc...
>>
>> Currently for jobs that are not run locally there is a general
>> environment_setup_file. There could be one optional environment_setup_file
>> for every job (or destination).
>>
>> In the end I created yet another wrapper.
>>
>> best,
>> ido
>>
>>
>> On Apr 25, 2014, at 2:34 PM, Björn Grüning 
>> wrote:
>>
>>> Hi Ido,
>>>
>>> I do not get your question in all detail, but it is possible to define a
>>> tool_dependencies.xml with a specific R version, + R libraries and use only
>>> that specific version from your tool with  tags.
>>>
>>> For an example please see various R tools from:
>>>
>>> https://github.com/bgruening/galaxytools
>>>
>>> Cheers,
>>> Bjoern
>>>
>>> Am 25.04.2014 14:25, schrieb Ido Tamir:

 Hi,
 has anything changed in galaxy in this regard?
 Any way to modify an environment before a tools is run?

 I now have a tool relying on R-devel and bioconductor devel, both of
 which I can load in a module.
 The tool comes from the toolshed with xml like:

 
 …


 I don't want to hack around in the tool itself, but simply load the
 necessary R-version.

 thank you very much,
 ido


 On Sep 13, 2013, at 3:23 AM, "Guest, Simon"
  wrote:

> Just been reading a bit more about the Galaxy packaging system.  Here's
> a slight modification to what I was suggesting that might fit in a bit
> better.  Apologies for not being more familiar with the existing system
> before proposing extensions.
>
> Recall that my goal is to support using a system-installed (native)
> package, at a defined version, which I aim to achieve by loading the
> appropriate environment module before running a tool.
>
> We still have tool_dependencies.xml defining a package at a particular
> version, but rather than download and build the source code, there's just 
> a
> directive that says how to pick up the correct program version at runtime,
> e.g. which environment module to load.
>
> So instead of the tool_dependencies.xml fragment:
> 
> 
> 
> 
>  type="download_by_url">http://downloads.sourceforge.net/project/bio-bwa/bwa-0.6.2.tar.bz2
> make
> 
> bwa
> $INSTALL_DIR/bin
> 
> 
>  action="prepend_to">$INSTALL_DIR/bin
> 
> 
> 
> 
> 
>
> We have something like this (NB: element and attribute names are for
> illustrative purposes only):
>
> 
> 
> 
> 
> bwa/0.6.2
> 
> 
> 
> 
>
> This causes the right thing (module load bwa/0.6.2) to be stuck into
> the dependenci

Re: [galaxy-dev] A proposed modules extension for toolshed wrappers

[Ido Tamir]

On Apr 25, 2014, at 3:55 PM, John Chilton  wrote:

> Additionally (perhaps more pertinently), I have an open pull request
> (created just this week) to allow setting environment variables on a
> per destination basis - this is sort of what you are getting at and
> might be a better way to go -
> https://bitbucket.org/galaxy/galaxy-central/pull-request/378/allow-specification-of-environment.
> With this modification you could hack up the path to reflect the
> changes made by the module load. That is a bit of hack but would work
> for this case. That said I think you make a good point about the
> environment file, I will modify the new  tags to allow a file
> attribute and just source that file. This will give you your per
> destination environment files - hopefully this is satisfactory.
> 
> 

That sounds great!

best,
ido

> 
> 
> On Fri, Apr 25, 2014 at 8:22 AM, Ido Tamir  wrote:
>> Dear  Björn,
>> 
>> maybe I could change the tool, which is not mine, and which I don't want to 
>> maintain to use a specific R-version that is already available on our 
>> cluster and
>> which I can put into my path with "module load R/3.1.0-devel" ( 
>> http://modules.sourceforge.net/)
>> 
>> there even is a requirements in the wrapper which is not fulfilled (these 
>> versions are not available at the moment) and still it installed without 
>> problems.
>>  
>>edgeR
>>limma
>>  
>> 
>> This requirements tags looks also rather inflexible to me. With an 
>> additional level of user configurable indirection it would be possible to 
>> make the tools
>> fit to different infrastructures without having to use binaries provided by 
>> somebody else, taking up space for just one tool etc...
>> 
>> Currently for jobs that are not run locally there is a general 
>> environment_setup_file. There could be one optional environment_setup_file 
>> for every job (or destination).
>> 
>> In the end I created yet another wrapper.
>> 
>> best,
>> ido
>> 
>> 
>> On Apr 25, 2014, at 2:34 PM, Björn Grüning  wrote:
>> 
>>> Hi Ido,
>>> 
>>> I do not get your question in all detail, but it is possible to define a 
>>> tool_dependencies.xml with a specific R version, + R libraries and use only 
>>> that specific version from your tool with  tags.
>>> 
>>> For an example please see various R tools from:
>>> 
>>> https://github.com/bgruening/galaxytools
>>> 
>>> Cheers,
>>> Bjoern
>>> 
>>> Am 25.04.2014 14:25, schrieb Ido Tamir:
 Hi,
 has anything changed in galaxy in this regard?
 Any way to modify an environment before a tools is run?
 
 I now have a tool relying on R-devel and bioconductor devel, both of which 
 I can load in a module.
 The tool comes from the toolshed with xml like:
 
 
 …
 
 
 I don't want to hack around in the tool itself, but simply load the 
 necessary R-version.
 
 thank you very much,
 ido
 
 
 On Sep 13, 2013, at 3:23 AM, "Guest, Simon"  
 wrote:
 
> Just been reading a bit more about the Galaxy packaging system.  Here's a 
> slight modification to what I was suggesting that might fit in a bit 
> better.  Apologies for not being more familiar with the existing system 
> before proposing extensions.
> 
> Recall that my goal is to support using a system-installed (native) 
> package, at a defined version, which I aim to achieve by loading the 
> appropriate environment module before running a tool.
> 
> We still have tool_dependencies.xml defining a package at a particular 
> version, but rather than download and build the source code, there's just 
> a directive that says how to pick up the correct program version at 
> runtime, e.g. which environment module to load.
> 
> So instead of the tool_dependencies.xml fragment:
> 
>   
>   
>   
>    type="download_by_url">http://downloads.sourceforge.net/project/bio-bwa/bwa-0.6.2.tar.bz2
>   make
>   
>   bwa
>   $INSTALL_DIR/bin
>   
>   
>    action="prepend_to">$INSTALL_DIR/bin
>   
>   
>   
>   
> 
> 
> We have something like this (NB: element and attribute names are for 
> illustrative purposes only):
> 
> 
>   
>   
>   
>   bwa/0.6.2
>   
>   
>   
> 
> 
> This causes the right thing (module load bwa/0.6.2) to be stuck into the 
> dependencies env.sh file when this package is installed from the 
> toolshed.  We could call this toolshed tool native_package_bwa_0_6_2, to 
> avoid confusion with the existing download-and-make one.
> 
> We might want a bit of flexibility on what actions are supported (in case 
> we want to support Software Collections, for example).
> 
> What do y

Re: [galaxy-dev] A proposed modules extension for toolshed wrappers

[Björn Grüning]

Hi Ida,

now I got it.
I may have an idea, not tested, but you could try the following.

- Add a file under $galaxy_tools/R_env/3.1.0/env.sh
- Create a symlink from $galaxy_tools/R_env/default/ to 
$galaxy_tools/R_env/3.1.0/

- change your requirement tag to
R_env

Now you can put in everything you want have in the env.sh file. That 
file will be sourced as soon as your tool will be executed. For example 
you can load your module.


Hope that works!
Thanks,
Bjoern


Dear  Björn,

maybe I could change the tool, which is not mine, and which I don't want to 
maintain to use a specific R-version that is already available on our cluster 
and
which I can put into my path with "module load R/3.1.0-devel" ( 
http://modules.sourceforge.net/)

there even is a requirements in the wrapper which is not fulfilled (these 
versions are not available at the moment) and still it installed without 
problems.
   
 edgeR
 limma
   

This requirements tags looks also rather inflexible to me. With an additional 
level of user configurable indirection it would be possible to make the tools
fit to different infrastructures without having to use binaries provided by 
somebody else, taking up space for just one tool etc...

Currently for jobs that are not run locally there is a general 
environment_setup_file. There could be one optional environment_setup_file for 
every job (or destination).

In the end I created yet another wrapper.

best,
ido


On Apr 25, 2014, at 2:34 PM, Björn Grüning  wrote:


Hi Ido,

I do not get your question in all detail, but it is possible to define a 
tool_dependencies.xml with a specific R version, + R libraries and use only that 
specific version from your tool with  tags.

For an example please see various R tools from:

https://github.com/bgruening/galaxytools

Cheers,
Bjoern

Am 25.04.2014 14:25, schrieb Ido Tamir:

Hi,
has anything changed in galaxy in this regard?
Any way to modify an environment before a tools is run?

I now have a tool relying on R-devel and bioconductor devel, both of which I 
can load in a module.
The tool comes from the toolshed with xml like:


…


I don't want to hack around in the tool itself, but simply load the necessary 
R-version.

thank you very much,
ido


On Sep 13, 2013, at 3:23 AM, "Guest, Simon"  
wrote:


Just been reading a bit more about the Galaxy packaging system.  Here's a 
slight modification to what I was suggesting that might fit in a bit better.  
Apologies for not being more familiar with the existing system before proposing 
extensions.

Recall that my goal is to support using a system-installed (native) package, at 
a defined version, which I aim to achieve by loading the appropriate 
environment module before running a tool.

We still have tool_dependencies.xml defining a package at a particular version, 
but rather than download and build the source code, there's just a directive 
that says how to pick up the correct program version at runtime, e.g. which 
environment module to load.

So instead of the tool_dependencies.xml fragment:




http://downloads.sourceforge.net/project/bio-bwa/bwa-0.6.2.tar.bz2
make

bwa
$INSTALL_DIR/bin


$INSTALL_DIR/bin






We have something like this (NB: element and attribute names are for 
illustrative purposes only):





bwa/0.6.2





This causes the right thing (module load bwa/0.6.2) to be stuck into the 
dependencies env.sh file when this package is installed from the toolshed.  We 
could call this toolshed tool native_package_bwa_0_6_2, to avoid confusion with 
the existing download-and-make one.

We might want a bit of flexibility on what actions are supported (in case we 
want to support Software Collections, for example).

What do you think?

cheers,
Simon

PS: In case it wasn't already clear, solving this problem well is quite 
important to us here at AgResearch.  ;-)


===
Attention: The information contained in this message and/or attachments
from AgResearch Limited is intended only for the persons or entities
to which it is addressed and may contain confidential and/or privileged
material. Any review, retransmission, dissemination or other use of, or
taking of any action in reliance upon, this information by persons or
entities other than the intended recipients is prohibited by AgResearch
Limited. If you have received this message in error, please notify the
sender immediately.
===

___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the 

Re: [galaxy-dev] A proposed modules extension for toolshed wrappers

[John Chilton]

Hello Ido,

Reading through your e-mail it is not clear to me whether the tool has
requirements that could be overridden. If it has package requirements
explicitly defined - those are completely configurable - you can write
arbitrary python plugins to resolve these any you wish - and (if the
modules are available on the galaxy server) there is an existing
plugin that can be used to resolve these packages as modules. I think
the best documentation of this may be on this pull request
(https://bitbucket.org/galaxy/galaxy-central/pull-request/228/tool-dependency-resolver-plugins-revision/diff).

Additionally (perhaps more pertinently), I have an open pull request
(created just this week) to allow setting environment variables on a
per destination basis - this is sort of what you are getting at and
might be a better way to go -
https://bitbucket.org/galaxy/galaxy-central/pull-request/378/allow-specification-of-environment.
With this modification you could hack up the path to reflect the
changes made by the module load. That is a bit of hack but would work
for this case. That said I think you make a good point about the
environment file, I will modify the new  tags to allow a file
attribute and just source that file. This will give you your per
destination environment files - hopefully this is satisfactory.

-John


On Fri, Apr 25, 2014 at 8:22 AM, Ido Tamir  wrote:
> Dear  Björn,
>
> maybe I could change the tool, which is not mine, and which I don't want to 
> maintain to use a specific R-version that is already available on our cluster 
> and
> which I can put into my path with "module load R/3.1.0-devel" ( 
> http://modules.sourceforge.net/)
>
> there even is a requirements in the wrapper which is not fulfilled (these 
> versions are not available at the moment) and still it installed without 
> problems.
>   
> edgeR
> limma
>   
>
> This requirements tags looks also rather inflexible to me. With an additional 
> level of user configurable indirection it would be possible to make the tools
> fit to different infrastructures without having to use binaries provided by 
> somebody else, taking up space for just one tool etc...
>
> Currently for jobs that are not run locally there is a general 
> environment_setup_file. There could be one optional environment_setup_file 
> for every job (or destination).
>
> In the end I created yet another wrapper.
>
> best,
> ido
>
>
> On Apr 25, 2014, at 2:34 PM, Björn Grüning  wrote:
>
>> Hi Ido,
>>
>> I do not get your question in all detail, but it is possible to define a 
>> tool_dependencies.xml with a specific R version, + R libraries and use only 
>> that specific version from your tool with  tags.
>>
>> For an example please see various R tools from:
>>
>> https://github.com/bgruening/galaxytools
>>
>> Cheers,
>> Bjoern
>>
>> Am 25.04.2014 14:25, schrieb Ido Tamir:
>>> Hi,
>>> has anything changed in galaxy in this regard?
>>> Any way to modify an environment before a tools is run?
>>>
>>> I now have a tool relying on R-devel and bioconductor devel, both of which 
>>> I can load in a module.
>>> The tool comes from the toolshed with xml like:
>>>
>>> 
>>> …
>>>
>>>
>>> I don't want to hack around in the tool itself, but simply load the 
>>> necessary R-version.
>>>
>>> thank you very much,
>>> ido
>>>
>>>
>>> On Sep 13, 2013, at 3:23 AM, "Guest, Simon"  
>>> wrote:
>>>
 Just been reading a bit more about the Galaxy packaging system.  Here's a 
 slight modification to what I was suggesting that might fit in a bit 
 better.  Apologies for not being more familiar with the existing system 
 before proposing extensions.

 Recall that my goal is to support using a system-installed (native) 
 package, at a defined version, which I aim to achieve by loading the 
 appropriate environment module before running a tool.

 We still have tool_dependencies.xml defining a package at a particular 
 version, but rather than download and build the source code, there's just 
 a directive that says how to pick up the correct program version at 
 runtime, e.g. which environment module to load.

 So instead of the tool_dependencies.xml fragment:
 



>>> type="download_by_url">http://downloads.sourceforge.net/project/bio-bwa/bwa-0.6.2.tar.bz2
make

bwa
$INSTALL_DIR/bin


>>> action="prepend_to">$INSTALL_DIR/bin




 

 We have something like this (NB: element and attribute names are for 
 illustrative purposes only):

 



bwa/0.6.2



 

 This causes the right thing (module load bwa/0.6.2) to be stuck into the 
 dependencies env.

Re: [galaxy-dev] "Intersect multiple sorted BED files" / tool-data/shared/ucsc/chrom/genome_test.len) could not be opened

[Sarah Maman]

Thanks Jennifer,

My problem is not solved yet.
The Galaxy version is not so old - November 2013 (* please see below)
len_file_path is already uncommented in universe.
Chromosome and genome are availables in loc files and 
tool-data/shared/ucsc/chrom also.


Best,
Sarah



Galaxy Version (*) :
galaxy-dev@genotoul /usr/local/bioinfo/src/galaxy-dev/galaxy-dist $ hg heads
changeset:   11219:5c789ab4144a
branch:  stable
tag: tip
user:Nate Coraor 
date:Mon Nov 04 15:04:42 2013 -0500
summary: Added tag release_2013.11.04 for changeset 26f58e05aa10

changeset:   11216:c458a0fe1ba8
parent:  11213:6d633418ecfa
parent:  11215:f79149dd3d35
user:Nate Coraor 
date:Mon Nov 04 14:56:57 2013 -0500
summary: Merge security fix for filtering tools from stable/next-stable.

changeset:   11168:654dfe820960
branch:  search
parent:  11166:776ca4a63a0b
user:Dannon Baker 
date:Fri Oct 25 15:32:18 2013 -0400
summary: Close search branch.

changeset:   11139:7286338fd77e
branch:  extract_genomic_dna_tool_enhancements
parent:  11137:7b014440dda1
user:John Chilton 
date:Fri Oct 25 00:09:25 2013 -0400
summary: Close branch extract_genomic_dna_tool_enhancements.

changeset:   11129:471f23116c5e
branch:  tool_api
parent:  11127:edab395a78b1
user:John Chilton 
date:Thu Oct 24 21:08:11 2013 -0500
summary: Close branch tool_api

changeset:   11126:e19d39296919
branch:  search
parent:  11124:35a847f0f33d
user:Dannon Baker 
date:Thu Oct 24 18:12:50 2013 -0400
summary: Re-close search feature branch.

changeset:   11087:28d3ef3d5ed5
branch:  search
parent:  11085:f82bbdc5c22b
user:John Chilton 
date:Tue Oct 22 01:47:05 2013 -0500
summary: Close branch search

changeset:   11017:c8bbd1d6c18c
branch:  api_copy_history
parent:  11015:138901353da9
user:Carl Eberhard 
date:Fri Oct 18 14:53:26 2013 -0400
summary: Close branch: api_copy_history

changeset:   11002:acf1a0062ea8
branch:  custom-toolbox2
parent:  10997:d816d6602e18
user:Dannon Baker 
date:Thu Oct 17 15:32:40 2013 -0400
summary: Close feature branch for toolbox filtering 2

changeset:   10968:eeebcddfa8f4
branch:  abrenner/histogram2xml-python-module-rpy-requires-1379010965302
parent:  10964:913f3ab84629
user:John Chilton 
date:Thu Oct 17 04:06:42 2013 +
summary: Close branch 
abrenner/histogram2xml-python-module-rpy-requires-1379010965302.


changeset:   10935:8b7e05aae54e
branch:  action_tpye_autoconf
parent:  10933:5a26cbd98609
user:Dave B. 
date:Tue Oct 15 15:21:52 2013 -0400
summary: Close branch action_tpye_autoconf

changeset:   10923:914cec313725
branch:  action_tpye_make_install
parent:  10921:be307f53645c
user:Dave Bouvier 
date:Tue Oct 15 11:32:42 2013 -0400
summary: Close branch action_tpye_make_install.

changeset:   10919:e3e016f0f6f3
branch:  R_environment
parent:  10917:e19c7819ea3c
user:Dave B. 
date:Tue Oct 15 11:24:21 2013 -0400
summary: Close branch R_environment

changeset:   10913:a2b2de536a47
branch:  action_type_misc
parent:  10911:223f968a3b3c
user:Dave B. 
date:Tue Oct 15 11:15:57 2013 -0400
summary: Close branch action_type_misc

changeset:   10612:e3e7b034cb0b
branch:  tool_api_submit
parent:  10608:abb889f8c5b7
user:Carl Eberhard 
date:Mon Sep 16 15:45:04 2013 -0400
summary: Close branch tool_api_submit

changeset:   10564:684ef8edd1d4
branch:  dataset-cleanup
parent:  10530:e4734c99812a
user:jeremy goecks 
date:Wed Sep 11 12:24:00 2013 -0400
summary: Close branch 10525:e4734c99812a

changeset:   10401:53c6769dcff4
branch:  cleanup_datasets_unicode_fix
parent:  10399:b245b46ec4c0
user:Dannon Baker 
date:Thu Aug 15 11:20:55 2013 -0400
summary: Branch close

changeset:   10364:e68eee8054d2
branch:  sort
parent:  10362:1d91e253734e
user:Dannon Baker 
date:Wed Aug 07 13:36:25 2013 -0400
summary: Branch close

changeset:   10346:a849924d6f9d
branch:  extended_metadata
parent:  10343:30f0aeacf88a
user:Dannon Baker 
date:Tue Aug 06 15:58:00 2013 -0400
summary: Close extended_metadata branch

changeset:   10132:10810850afaa
branch:  list_published_export
parent:  10129:1d71126419b2
user:jeremy goecks 
date:Fri Jun 21 09:27:35 2013 -0400
summary: Close list_published_export branch

changeset:   10119:6090712e7f29
branch:  provenance
parent:  10115:471457199db2
user:Dannon Baker 
date:Thu Jun 20 12:58:16 2013 -0400
summary: Close provenance branch

changeset:   10114:49771dac79f2
branch:  provenance
parent:  10108:0bb601dceb65
user:Dannon Baker 
date:Thu Jun 2

Re: [galaxy-dev] A proposed modules extension for toolshed wrappers

[Ido Tamir]

Dear  Björn,

maybe I could change the tool, which is not mine, and which I don't want to 
maintain to use a specific R-version that is already available on our cluster 
and
which I can put into my path with "module load R/3.1.0-devel" ( 
http://modules.sourceforge.net/)

there even is a requirements in the wrapper which is not fulfilled (these 
versions are not available at the moment) and still it installed without 
problems.
  
edgeR
limma
  

This requirements tags looks also rather inflexible to me. With an additional 
level of user configurable indirection it would be possible to make the tools
fit to different infrastructures without having to use binaries provided by 
somebody else, taking up space for just one tool etc...

Currently for jobs that are not run locally there is a general 
environment_setup_file. There could be one optional environment_setup_file for 
every job (or destination).

In the end I created yet another wrapper.

best,
ido


On Apr 25, 2014, at 2:34 PM, Björn Grüning  wrote:

> Hi Ido,
> 
> I do not get your question in all detail, but it is possible to define a 
> tool_dependencies.xml with a specific R version, + R libraries and use only 
> that specific version from your tool with  tags.
> 
> For an example please see various R tools from:
> 
> https://github.com/bgruening/galaxytools
> 
> Cheers,
> Bjoern
> 
> Am 25.04.2014 14:25, schrieb Ido Tamir:
>> Hi,
>> has anything changed in galaxy in this regard?
>> Any way to modify an environment before a tools is run?
>> 
>> I now have a tool relying on R-devel and bioconductor devel, both of which I 
>> can load in a module.
>> The tool comes from the toolshed with xml like:
>> 
>> 
>> …
>> 
>> 
>> I don't want to hack around in the tool itself, but simply load the 
>> necessary R-version.
>> 
>> thank you very much,
>> ido
>> 
>> 
>> On Sep 13, 2013, at 3:23 AM, "Guest, Simon"  
>> wrote:
>> 
>>> Just been reading a bit more about the Galaxy packaging system.  Here's a 
>>> slight modification to what I was suggesting that might fit in a bit 
>>> better.  Apologies for not being more familiar with the existing system 
>>> before proposing extensions.
>>> 
>>> Recall that my goal is to support using a system-installed (native) 
>>> package, at a defined version, which I aim to achieve by loading the 
>>> appropriate environment module before running a tool.
>>> 
>>> We still have tool_dependencies.xml defining a package at a particular 
>>> version, but rather than download and build the source code, there's just a 
>>> directive that says how to pick up the correct program version at runtime, 
>>> e.g. which environment module to load.
>>> 
>>> So instead of the tool_dependencies.xml fragment:
>>> 
>>>
>>>
>>>
>>>>> type="download_by_url">http://downloads.sourceforge.net/project/bio-bwa/bwa-0.6.2.tar.bz2
>>>make
>>>
>>>bwa
>>>$INSTALL_DIR/bin
>>>
>>>
>>>>> action="prepend_to">$INSTALL_DIR/bin
>>>
>>>
>>>
>>>
>>> 
>>> 
>>> We have something like this (NB: element and attribute names are for 
>>> illustrative purposes only):
>>> 
>>> 
>>>
>>>
>>>
>>>bwa/0.6.2
>>>
>>>
>>>
>>> 
>>> 
>>> This causes the right thing (module load bwa/0.6.2) to be stuck into the 
>>> dependencies env.sh file when this package is installed from the toolshed.  
>>> We could call this toolshed tool native_package_bwa_0_6_2, to avoid 
>>> confusion with the existing download-and-make one.
>>> 
>>> We might want a bit of flexibility on what actions are supported (in case 
>>> we want to support Software Collections, for example).
>>> 
>>> What do you think?
>>> 
>>> cheers,
>>> Simon
>>> 
>>> PS: In case it wasn't already clear, solving this problem well is quite 
>>> important to us here at AgResearch.  ;-)
>>> 
>>> 
>>> ===
>>> Attention: The information contained in this message and/or attachments
>>> from AgResearch Limited is intended only for the persons or entities
>>> to which it is addressed and may contain confidential and/or privileged
>>> material. Any review, retransmission, dissemination or other use of, or
>>> taking of any action in reliance upon, this information by persons or
>>> entities other than the intended recipients is prohibited by AgResearch
>>> Limited. If you have received this message in error, please notify the
>>> sender immediately.
>>> ===
>>> 
>>> ___
>>> Please keep all replies on the list by using "reply all"
>>> in your mail client.  To manage your subscriptions to this
>>> and other Galaxy lists, please use the interface at:
>>>  http://lists.bx.psu.edu/
>>

Re: [galaxy-dev] A proposed modules extension for toolshed wrappers

[Björn Grüning]

Hi Ido,

I do not get your question in all detail, but it is possible to define a 
tool_dependencies.xml with a specific R version, + R libraries and use 
only that specific version from your tool with  tags.


For an example please see various R tools from:

https://github.com/bgruening/galaxytools

Cheers,
Bjoern

Am 25.04.2014 14:25, schrieb Ido Tamir:

Hi,
has anything changed in galaxy in this regard?
Any way to modify an environment before a tools is run?

I now have a tool relying on R-devel and bioconductor devel, both of which I 
can load in a module.
The tool comes from the toolshed with xml like:


…


I don't want to hack around in the tool itself, but simply load the necessary 
R-version.

thank you very much,
ido


On Sep 13, 2013, at 3:23 AM, "Guest, Simon"  
wrote:


Just been reading a bit more about the Galaxy packaging system.  Here's a 
slight modification to what I was suggesting that might fit in a bit better.  
Apologies for not being more familiar with the existing system before proposing 
extensions.

Recall that my goal is to support using a system-installed (native) package, at 
a defined version, which I aim to achieve by loading the appropriate 
environment module before running a tool.

We still have tool_dependencies.xml defining a package at a particular version, 
but rather than download and build the source code, there's just a directive 
that says how to pick up the correct program version at runtime, e.g. which 
environment module to load.

So instead of the tool_dependencies.xml fragment:




http://downloads.sourceforge.net/project/bio-bwa/bwa-0.6.2.tar.bz2
make

bwa
$INSTALL_DIR/bin


$INSTALL_DIR/bin






We have something like this (NB: element and attribute names are for 
illustrative purposes only):





bwa/0.6.2





This causes the right thing (module load bwa/0.6.2) to be stuck into the 
dependencies env.sh file when this package is installed from the toolshed.  We 
could call this toolshed tool native_package_bwa_0_6_2, to avoid confusion with 
the existing download-and-make one.

We might want a bit of flexibility on what actions are supported (in case we 
want to support Software Collections, for example).

What do you think?

cheers,
Simon

PS: In case it wasn't already clear, solving this problem well is quite 
important to us here at AgResearch.  ;-)


===
Attention: The information contained in this message and/or attachments
from AgResearch Limited is intended only for the persons or entities
to which it is addressed and may contain confidential and/or privileged
material. Any review, retransmission, dissemination or other use of, or
taking of any action in reliance upon, this information by persons or
entities other than the intended recipients is prohibited by AgResearch
Limited. If you have received this message in error, please notify the
sender immediately.
===

___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  http://lists.bx.psu.edu/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/



___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
   http://lists.bx.psu.edu/

To search Galaxy mailing lists use the unified search at:
   http://galaxyproject.org/search/mailinglists/


___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
 http://lists.bx.psu.edu/

To search Galaxy mailing lists use the unified search at:
 http://galaxyproject.org/search/mailinglists/

Re: [galaxy-dev] A proposed modules extension for toolshed wrappers

[Ido Tamir]

Hi,
has anything changed in galaxy in this regard?
Any way to modify an environment before a tools is run?

I now have a tool relying on R-devel and bioconductor devel, both of which I 
can load in a module.
The tool comes from the toolshed with xml like:

 
…


I don't want to hack around in the tool itself, but simply load the necessary 
R-version.

thank you very much,
ido


On Sep 13, 2013, at 3:23 AM, "Guest, Simon"  
wrote:

> Just been reading a bit more about the Galaxy packaging system.  Here's a 
> slight modification to what I was suggesting that might fit in a bit better.  
> Apologies for not being more familiar with the existing system before 
> proposing extensions.
> 
> Recall that my goal is to support using a system-installed (native) package, 
> at a defined version, which I aim to achieve by loading the appropriate 
> environment module before running a tool.
> 
> We still have tool_dependencies.xml defining a package at a particular 
> version, but rather than download and build the source code, there's just a 
> directive that says how to pick up the correct program version at runtime, 
> e.g. which environment module to load.
> 
> So instead of the tool_dependencies.xml fragment:
> 
>
>
>
> type="download_by_url">http://downloads.sourceforge.net/project/bio-bwa/bwa-0.6.2.tar.bz2
>make
>
>bwa
>$INSTALL_DIR/bin
>
>
> action="prepend_to">$INSTALL_DIR/bin
>
>
>
>
> 
> 
> We have something like this (NB: element and attribute names are for 
> illustrative purposes only):
> 
> 
>
>
>
>bwa/0.6.2
>
>
>
> 
> 
> This causes the right thing (module load bwa/0.6.2) to be stuck into the 
> dependencies env.sh file when this package is installed from the toolshed.  
> We could call this toolshed tool native_package_bwa_0_6_2, to avoid confusion 
> with the existing download-and-make one.
> 
> We might want a bit of flexibility on what actions are supported (in case we 
> want to support Software Collections, for example).
> 
> What do you think?
> 
> cheers,
> Simon
> 
> PS: In case it wasn't already clear, solving this problem well is quite 
> important to us here at AgResearch.  ;-)
> 
> 
> ===
> Attention: The information contained in this message and/or attachments
> from AgResearch Limited is intended only for the persons or entities
> to which it is addressed and may contain confidential and/or privileged
> material. Any review, retransmission, dissemination or other use of, or
> taking of any action in reliance upon, this information by persons or
> entities other than the intended recipients is prohibited by AgResearch
> Limited. If you have received this message in error, please notify the
> sender immediately.
> ===
> 
> ___
> Please keep all replies on the list by using "reply all"
> in your mail client.  To manage your subscriptions to this
> and other Galaxy lists, please use the interface at:
>  http://lists.bx.psu.edu/
> 
> To search Galaxy mailing lists use the unified search at:
>  http://galaxyproject.org/search/mailinglists/


___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  http://lists.bx.psu.edu/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/

[galaxy-dev] "Intersect multiple sorted BED files" / tool-data/shared/ucsc/chrom/genome_test.len) could not be opened

[Jennifer Jackson]

Hello Sarah,

This commit was made some time ago and is already in the distribution. 
Are you running the latest distribution?


And you have uncommented the configuration line for this in your 
universe.wgsi.ini file?


# Directory where chrom len files are kept, currently mainly used by 
trackster

#len_file_path = tool-data/shared/ucsc/chrom  <--- here

Upgrade and adjust the file as needed.
https://wiki.galaxyproject.org/News/2014_04_14_Galaxy_Distribution

I am going to forward this over to the galaxy-...@bx.psu.edu mailing 
list for further troubleshooting, should this not resolve the problem. 
Please keep all replies on that list.


Best,

Jen
Galaxy team
https://wiki.galaxyproject.org/MailingLists

*Galaxy Biostar is replacing the galaxy-user mailing list! /Update now! /**
**   Here's how ->>> https://wiki.galaxyproject.org/Support/Biostar**
*

On 4/22/14 3:25 AM, Sarah Maman wrote:

Hello,

I try to use "Intersect multiple sorted BED files" tool without success.
A genome_test have been added in galaxy (genome_test.len in 
galaxy-dist/tool-data/shared/ucsc/chrom/, .fa and .dict added in .loc 
fils in tool-data/) but "Intersect multiple sorted BED files" gives 
this error message :
An error occurred running this job: /Epilog : job finished at lun. 
mars 24 15:55:04 CET 2014
Error: The requested genome file 
(//galaxy-dist/tool-data/shared/ucsc/chrom/genome_test.len) could 
not be opened. Exiting!


So, /I have integrated the code modification submitted in Galaxy 
Central 
(https://bitbucket.org/galaxy/galaxy-central/commits/7105c53139d4b8649e6a3714bc117118989712a2):

-chrom_info = os.path.join( trans.app.config.tool_data_path, 
'shared','ucsc','chrom', "%s.len" % input_dbkey )
+chrom_info = os.path.join( trans.app.config.len_file_path, 
"%s.len" % input_dbkey )

But I always have the same error message.

Could you please help us ?
Thanks in advance,
Sarah

--
   --*--
Sarah Maman
INRA - GenPhySE - SIGENAE
http://www.sigenae.org/
Chemin de Borde-Rouge - Auzeville - BP 52627
31326 Castanet-Tolosan cedex - FRANCE
Tel:   +33(0)5.61.28.57.08
Fax:   +33(0)5.61.28.57.53
  --*--


___
The Galaxy User list should be used for the discussion of
Galaxy analysis and other features on the public server
at usegalaxy.org.  Please keep all replies on the list by
using "reply all" in your mail client.  For discussion of
local Galaxy instances and the Galaxy source code, please
use the Galaxy Development list:

   http://lists.bx.psu.edu/listinfo/galaxy-dev

To manage your subscriptions to this and other Galaxy lists,
please use the interface at:

   http://lists.bx.psu.edu/

To search Galaxy mailing lists use the unified search at:

   http://galaxyproject.org/search/mailinglists/


--
Jennifer Hillman-Jackson
http://galaxyproject.org

___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  http://lists.bx.psu.edu/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/

Re: [galaxy-dev] AWS CloudFormation for Galaxy ?

[Enis Afgan]

Someone else might have but we have not (the same argument still holds).

Please let us know if you go down that path,
Enis


On Fri, Apr 25, 2014 at 12:35 PM, Luca Toldo  wrote:

> dear Galaxians,
> AWS CloudFormation is now mature, and I have interest in exploiting it for
> extending my local grid installation with SGE-based execute nodes with
> galaxy on top.
>
> This is something that Enis and Ryan already discussed more than 3 years
> ago
> https://www.mail-archive.com/galaxy-dev@lists.bx.psu.edu/msg00427.html
>
> however I couldn't find any followup to that discussion.
>
> Has anyone already done such script ?
>
>
>
___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  http://lists.bx.psu.edu/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/

Re: [galaxy-dev] Galaxy - Cloudman

[Dannon Baker]

Do you have Cloudman admin access on this instance?

The first thing I'd check would be http:///cloud to verify
that your instance has free disk space and to look for any errors in the
log.  Then, check out the admin service list
(http:///cloud/admin)
to see if there are any service issues.  Next to the SGE service, you
should see a 'qstat' link that'll show you the status of any jobs submitted
to the cluster.  If you're not running any extra worker nodes, check this
page to verify that the master is set to run jobs.

If you look through this stuff and nothing looks obvious, feel free to ping
me off list and I can try to take a look with you.

-Dannon


On Fri, Apr 25, 2014 at 2:18 AM, Alessia Deglincerti <
adeglin...@mail.rockefeller.edu> wrote:

> Hey Dannon,
>
> I am using Galaxy (Cloudman) to analyze RNA seq data. I have a bunch of
> datasets that I ran TopHat on. I am now trying to run Cufflinks etc. I used
> the same data and workflow yesterday to run Cufflinks etc and that worked
> fine. I used my own reference annotation (Homo sapiens hg38) that I
> imported from UCSC but I wanted to compare to some older data I have and so
> now I  trying to run it on hg17. I also have some other datasets in the
> queue. All my jobs are gray and have been waiting to ran for the past 24+
> hours. I am seeing the error as I am waiting for my jobs to run, I get a
> notification at the top of my queue/history saying that there was a problem
> getting updates and to contact a site administrator if the problem persists
> (I've seen the notification a few times) and when I click the "details"
> link in the notification it prompts me with the details pasted below.
>
> I am using all of the default options for my jobs and the same workflow
> has been working fine for me in the past on the same datasets.
>
> Thanks for any help you can give me!
>
> Alessia
>
>
>
> On Apr 24, 2014, at 10:24 PM, Dannon Baker  wrote:
>
> Hey Alessia,
>
> I'd love to help, but I need a little more context here.  Where are you
> seeing this error, what exactly are you doing when it appears, and can you
> tell me a little bit more about the Galaxy configuration you're working
> with?
>
> -Dannon
>
>
> On Thu, Apr 24, 2014 at 8:37 PM, Jennifer Jackson  wrote:
>
>>  Posting to the galaxy-...@bx.psu.edu mailing list.
>> https://wiki.galaxyproject.org/MailingLists
>>
>>  My instance of Galaxy seems to be stuck (same place since last night)
>> and I have received the error message several times now.
>>
>>  Pasting the error message below:
>>
>>  *Details*
>>
>> user
>>
>> username
>>
>> adeglincer
>>
>> quota_percent
>>
>> 68
>>
>> total_disk_usage
>>
>> 184947273683
>>
>> nice_total_disk_usage
>>
>> 172.2 GB
>>
>> email
>>
>> xxx
>>
>> is_admin
>>
>> false
>>
>> tags_used
>>
>> model_class
>>
>> User
>>
>> id
>>
>> xxx
>>
>> source
>>
>> HDACollection(xxx)
>>
>> xhr
>>
>> readyState
>>
>> 4
>>
>> responseText
>>
>> {"err_msg": "Uncaught exception in exposed API method:", "err_code": 0}
>>
>> responseJSON
>>
>> err_msg
>>
>> Uncaught exception in exposed API method:
>>
>> err_code
>>
>> 0
>>
>> status
>>
>> 500
>>
>> statusText
>>
>> Internal Server Error
>>
>> responseHeaders
>>
>> Date
>>
>> Thu, 24 Apr 2014 20:51:39 GMT
>>
>> cache-control
>>
>> max-age=0,no-cache,no-store
>>
>> Server
>>
>> nginx/1.4.7
>>
>> Connection
>>
>> keep-alive
>>
>> Transfer-Encoding
>>
>> chunked
>>
>> Content-Type
>>
>> application/json
>>
>> options
>>
>> data
>>
>> parse
>>
>> true
>>
>> emulateHTTP
>>
>> false
>>
>> emulateJSON
>>
>> false
>>
>>
>> ___
>> Please keep all replies on the list by using "reply all"
>> in your mail client.  To manage your subscriptions to this
>> and other Galaxy lists, please use the interface at:
>>   http://lists.bx.psu.edu/
>>
>> To search Galaxy mailing lists use the unified search at:
>>   http://galaxyproject.org/search/mailinglists/
>>
>
>
___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  http://lists.bx.psu.edu/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/

[galaxy-dev] AWS CloudFormation for Galaxy ?

[Luca Toldo]

dear Galaxians,
AWS CloudFormation is now mature, and I have interest in exploiting it for
extending my local grid installation with SGE-based execute nodes with
galaxy on top.

This is something that Enis and Ryan already discussed more than 3 years
ago
https://www.mail-archive.com/galaxy-dev@lists.bx.psu.edu/msg00427.html

however I couldn't find any followup to that discussion.

Has anyone already done such script ?
___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  http://lists.bx.psu.edu/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/

Re: [galaxy-dev] show pie of html output krona (in eye mode)

[Stef van Lieshout]

This might have to do with the fact that galaxy by default sanitizes
html, you can alter this behaviour with an option in universe_wsgi.ini
(but might compromise security):

# Sanitize All HTML Tool Output
# By default, all tool output served as 'text/html' will be sanitized
# thoroughly. This can be disabled if you have special tools that
require
# unaltered output.
sanitize_all_html = True/False

Gr,
Stef


- Original message -
From: Clement Lionnet 
To: galaxy-dev@lists.bx.psu.edu
Subject: [galaxy-dev] show pie of html output krona (in eye mode)
Date: Fri, 25 Apr 2014 10:18:16 +0200

Hi all,

I have a problem when I would to show the pie of krona. Galaxy show the
data but not the pie. Is it possible to show the pie in the eye mode?

I think that the jquery script which is in the output, is not read by
Galaxy.

Thanks,
Clément 

___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  http://lists.bx.psu.edu/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/

___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  http://lists.bx.psu.edu/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/

Re: [galaxy-dev] workflow_execute_parameters.py

[Olivia Doppelt-Azeroual]

Dear Dannon,

Thank you for your answer, it works better now that I know the steps ID 
for my workflow.


I have another question though.

The second step of my WF uses a parameter input named "prefix". I really 
need to specify it when I launch the WF using the API.


When i display the WF using the url you gave me, the parameters prefix, 
is not there and when I use this in my command line, the specified Value 
is not read by Galaxy.


python workflow_execute_parameters.py 43359c833b8977b01d6237d9b839dc27 
http://127.0.0.1:8080/api/workflows f2db41e1fa331b3e a799d38679e985db 
'10=hda=52ea4fd71e9f7c3f' 'param=export_sbw=prefix=test'


Could you help me on that ? How do I give a value to a parameter if it 
does not appear in the WF description ??


Thanks in advance,
--
Olivia


Le 23/04/14 18:40, Dannon Baker a écrit :

Hi Olivia!

It's probably talking about the "workflowstep" object with identifier 
8.  If you request the description of this workflow via the API 
(http://yourgalaxyinstance.org/api/workflows/workflow>"), you'll be given the exact step identifiers to use.  In 
other words, unless this is the first workflow ever created on a brand 
new galaxy instance, your step ids will not be 0 and 1.


-Dannon


On Wed, Apr 23, 2014 at 12:30 PM, Olivia Doppelt-Azeroual 
mailto:olivia.dopp...@pasteur.fr>> wrote:


Dear developpers,

I'm trying to launch a small workflow using the api script
workflow_execute_parameters.py

I have 2 steps: the input data step and a second step which needs
a parameter called prefix.


The workflow is attached to this email.

I get an error on the parameters inputs which asks me to specify
information about an 8th step (which does not exist)


Is there a test case available to use this script ?

Please keep me informed on that.

Have a nice day,

-- 
Olivia Doppelt-Azeroual, PhD

Tel: 92 15
CIB - Institut Pasteur


___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
http://lists.bx.psu.edu/

To search Galaxy mailing lists use the unified search at:
http://galaxyproject.org/search/mailinglists/





--
Olivia Doppelt-Azeroual, PhD
Tel: 92 15
CIB - Institut Pasteur

___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  http://lists.bx.psu.edu/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/

[galaxy-dev] show pie of html output krona (in eye mode)

[Clement Lionnet]

Hi all,

I have a problem when I would to show the pie of krona. Galaxy show the data 
but not the pie. Is it possible to show the pie in the eye mode?

I think that the jquery script which is in the output, is not read by Galaxy.

Thanks,
Clément 

___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  http://lists.bx.psu.edu/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/