Re: [galaxy-dev] How to count the number of Workflow steps?

[Richard Park]

Hi Dannon,
Thanks for the response. If there is a "RenameDatasetActionoutput_file"
post job action, does that implicity mean that all other output files are
considered HideDatasetActions?
thanks,
Richard

On Tue, Mar 5, 2013 at 6:36 PM, Dannon Baker  wrote:

> Manually adding "HideDatasetActions" via the right panel in the editor was
> deprecated a while back in favor of the workflow output 'asterisks' you see
> rendered on each step in the workflow editor, but they're still used behind
> the scenes.  When you flag a step as an output, all other non-output steps
> should have HideDatasetActions added to the step automatically.
>
> -Dannon
>
>
>
> On Tue, Mar 5, 2013 at 5:32 PM, Richard Park  wrote:
>
>> Hi everyone,
>> Does anybody know how Galaxy currently determines which outputs from a
>> tool are shown when a workflow is run? I'm currently dynamically creating
>> workflows based on the number of given inputs and iterating through all
>> "HideDatasetAction*" for each tools "post_job_actions" to figure out the
>> total number of steps that will be run.
>>
>> "post_job_actions": {
>> "HideDatasetActionoutput": {
>> "action_arguments": {},
>> "action_type": "HideDatasetAction",
>> "output_name": "output"
>> },
>> "HideDatasetActionoutput_suppressed_reads_l": {
>> "action_arguments": {},
>> "action_type": "HideDatasetAction",
>> "output_name": "output_suppressed_reads_l"
>> },
>> "HideDatasetActionoutput_suppressed_reads_r": {
>> "action_arguments": {},
>> "action_type": "HideDatasetAction",
>> "output_name": "output_suppressed_reads_r"
>> },
>> "HideDatasetActionoutput_unmapped_reads_l": {
>> "action_arguments": {},
>> "action_type": "HideDatasetAction",
>> "output_name": "output_unmapped_reads_l"
>> },
>> "HideDatasetActionoutput_unmapped_reads_r": {
>> "action_arguments": {},
>> "action_type": "HideDatasetAction",
>> "output_name": "output_unmapped_reads_r"
>> }
>> },
>>
>> In the latest galaxy, there no longer seems to be
>> any "HideDatasetAction*" in each tools "post_job_actions" options. How does
>> Galaxy know which output files for a tool it should show or process?
>>
>> Does it have to do something with "RenameDatasetAction*"?
>>
>> "post_job_actions": {
>> "RenameDatasetActionoutput_file": {
>> "action_arguments": {
>> "newname": "19_output_file"
>> },
>> "action_type": "RenameDatasetAction",
>> "output_name": "output_file"
>> }
>> },
>>
>> Any guidance would be much appreciated.
>> Thanks,
>>
>> ___
>> Please keep all replies on the list by using "reply all"
>> in your mail client.  To manage your subscriptions to this
>> and other Galaxy lists, please use the interface at:
>>
>>   http://lists.bx.psu.edu/
>>
>
>
___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:

  http://lists.bx.psu.edu/

[galaxy-dev] Validating dynamic inputs stream of consciousness

[John Chilton]

I have been extending the LWR so that one can now stand up public LWR
servers (https://lwr.readthedocs.org/en/latest/#setting-up-a-public-lwr-server),
the idea is that you can publicly share data and computation with any
Galaxy instances in the world easily and seamlessly via vanilla Galaxy
tools. This could serve the foundation of a distributed service
architecture where tools are the contracts, Galaxy instances the
clients, and LWRs the server containers.

One thing that did need to change with the LWR is that inputs need to
be validated. It would obviously not be a good idea to allow arbitrary
command-line or script executions to unauthenticated clients.

This was accomplished in part by allowing you to configure a Galaxy
toolbox or tool_conf file for the LWR. The LWR reads validation logic
for command-line and configfiles from the XML and verifies them before
execution.

The test cases I used to build up this validation extension to the
tool XML can be found here:

https://bitbucket.org/jmchilton/lwr/src/tip/test/validator_test.py

Question 1: Would Galaxy benefit from implementing this validation
stuff in the core framework as well? I think of myself as fairly
security conscious and my sense is no, but it could I see the
argument. There are potentially complex interactions between user
inputs and cheetah templates that one might sleep better knowing
everything is being validated before execution (the user didn't cause
the cheetah template to somehow render "; rm -rf /").

Question 2: Looking back on the validation stuff, it seems I am just
rewriting in XML what cheetah is doing. The duplication makes me think
the validation stuff might serve the foundation for a better (or at
least more secure) way to build up commands.


  tophat_wrapper.py
  

  
  
 $refGenomeSource.ownFile
   

The client could easily compile that into a cheetah template to build
the command, the server could compile it into a regular expression to
validate the input.

These aren't things that pressingly need to be added to the core
Galaxy framework, but things to think about longer term. If there is
any interest from the core Galaxy team I would be happy to implement
any of this with any desired changes.

Thanks all,
-John


John Chilton
Senior Software Developer
University of Minnesota Supercomputing Institute
Office: 612-625-0917
Cell: 612-226-9223
https://twitter.com/jmchilton
https://bitbucket.org/jmchilton
https://github.com/jmchilton
___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:

  http://lists.bx.psu.edu/

Re: [galaxy-dev] Built-in reference genome

[Ross]

No, you need to follow the documentation in the relevant sample loc file -
not name individual index files.

(Your loc file had already been edited then. Be warned that if the
bwa_index entry in tool_data_table.xml file has also been hacked to change
the field definitions, all bets are off)

The bwa_index.loc created the first time a fresh Galaxy checkout is run is
copied from tool-data/bwa_index.loc.sample - there you'll find the
documentation, which suggests that

ecoli\tecoli\tecoliLongNameDescription\t/Users/charlesmurphy/Desktop/Research/data/genome/ecoli/bwa/ecoli.fa

should work after a restart and a fresh reload of the tool in your browser
so the new data are loaded from the loc file - assuming the Galaxy user has
permission to read those index files of course.

If not, it's time to read the suggestions in the wiki about providing more
details when posting questions to the list!
eg what Galaxy release (eg hg heads), OS, platform, python version etc


On Wed, Mar 6, 2013 at 12:59 PM, Charles Murphy
wrote:

> Hi,
>
> So I removed the edit I made to the builds.txt file. I' am not sure I
> completely understand how you think I should edit the
> tool-data/bwa_index.loc file. Previously it contained only the following
> line (tab delimited):
>
> ecoli   ecoli   ecoli
> /Users/charlesmurphy/Desktop/Research/data/genome/ecoli/bwa/ecoli.fa
>
> When I open the Map with BWA for Illumina tool in Galaxy, there are no
> reference genomes to select from when I have "Use a built-in index"
> selected.
>
> I've previously examined the comments in the tool-data/bwa_index.loc file.
> I interpreted that I need one line in the tool-data/bwa_index.loc file for
> each "index set". Just to be clear, you aren't suggesting that I add five
> additional lines for the files with the .amb, .ann, .bwt, .pac, and .sa
> extensions? So my tool-data/bwa_index.loc file contains:
>
> ecoli   ecoli   ecoli
> /Users/charlesmurphy/Desktop/Research/data/genome/ecoli/bwa/ecoli.fa
> ecoli   ecoli   ecoli
> /Users/charlesmurphy/Desktop/Research/data/genome/ecoli/bwa/ecoli.fa.amb
> ecoli   ecoli   ecoli
> /Users/charlesmurphy/Desktop/Research/data/genome/ecoli/bwa/ecoli.fa.ann
> ecoli   ecoli   ecoli
> /Users/charlesmurphy/Desktop/Research/data/genome/ecoli/bwa/ecoli.fa.bwt
> ecoli   ecoli   ecoli
> /Users/charlesmurphy/Desktop/Research/data/genome/ecoli/bwa/ecoli.fa.pac
> ecoli   ecoli   ecoli
> /Users/charlesmurphy/Desktop/Research/data/genome/ecoli/bwa/ecoli.fa.sa
>
> I tried that and it does not work. Your response is appreciated.
>
> Charlie
>
>
> On Tue, Mar 5, 2013 at 7:11 PM, Ross  wrote:
>
>> Hi, Charles,
>>
>> builds.txt is what tells the Galaxy framework about available genomes.
>> Various binary index files created from those genomes are separately
>> managed for specific tools since some folks won't need some application
>> specific sets of indexes for any given genome - so you need to edit a
>> different file!
>>
>> The BWA for illumina tool for example gets BWA index files from
>> tool-data/bwa_index.loc so that's where the reference to your ecoli.fa and
>> related BWA index files needs to be added - the format is described in the
>> comments you'll find when you edit that file. Tabs for delimiters again and
>> another restart should fix your problem for that tool?
>>
>>
>>
>>
>>
>>
>> On Wed, Mar 6, 2013 at 11:46 AM, Charles Murphy <
>> murphy.charl...@gmail.com> wrote:
>>
>>> Hi,
>>>
>>> I' am trying to add built-in reference genomes for my local galaxy
>>> installation. I know this topic has been covered multiple times: I have
>>> looked through the discussion threads and the wiki (e.g. NGS Local Setup),
>>> but I' am still unsuccessful. Previously I have just loaded a reference
>>> genome through the history, but in the interest of saving time and space
>>> while I experiment with Galaxy I wish to use the built-in feature.
>>>
>>> Here is what I have tried:
>>>
>>> 1. Indexed ecoli.fa using bwa and put all files (fasta and index files)
>>> under /Users/charlesmurphy/Desktop/Research/data/genome/ecoli/bwa/
>>>
>>> The directory looks like:
>>> -rwxrwxrwx@ 1 charlesmurphy  staff  4706043 Mar  5 14:54 ecoli.fa
>>> -rwxrwxrwx  1 charlesmurphy  staff   12 Mar  5 16:38 ecoli.fa.amb
>>> -rwxrwxrwx  1 charlesmurphy  staff  111 Mar  5 16:38 ecoli.fa.ann
>>> -rwxrwxrwx  1 charlesmurphy  staff  4639752 Mar  5 16:38 ecoli.fa.bwt
>>> -rwxrwxrwx  1 charlesmurphy  staff  1159920 Mar  5 16:38 ecoli.fa.pac
>>> -rwxrwxrwx  1 charlesmurphy  staff  2319888 Mar  5 16:38 ecoli.fa.sa
>>>
>>> 2. Downloaded and installed the bwa wrapper files from the Galaxy main
>>> toolshed, and put bwa in $PATH.
>>> 3. Added the following line to the bwa_index.loc (using tabs as the
>>> delimiter):
>>> ecoli   ecoli   ecoli
>>> /Users/charlesmurphy/Desktop/Research/data/genome/ecoli/bwa/ecoli.fa
>>> 4. Added the following line to builds.txt (using tabs as the delimiter):
>>> ecoli ecoli
>>> 5. Restarted Galaxy.
>>>
>>> The ecoli genome does not show u

Re: [galaxy-dev] Built-in reference genome

[Charles Murphy]

Hi,

So I removed the edit I made to the builds.txt file. I' am not sure I
completely understand how you think I should edit the
tool-data/bwa_index.loc file. Previously it contained only the following
line (tab delimited):

ecoli   ecoli   ecoli
/Users/charlesmurphy/Desktop/Research/data/genome/ecoli/bwa/ecoli.fa

When I open the Map with BWA for Illumina tool in Galaxy, there are no
reference genomes to select from when I have "Use a built-in index"
selected.

I've previously examined the comments in the tool-data/bwa_index.loc file.
I interpreted that I need one line in the tool-data/bwa_index.loc file for
each "index set". Just to be clear, you aren't suggesting that I add five
additional lines for the files with the .amb, .ann, .bwt, .pac, and .sa
extensions? So my tool-data/bwa_index.loc file contains:

ecoli   ecoli   ecoli
/Users/charlesmurphy/Desktop/Research/data/genome/ecoli/bwa/ecoli.fa
ecoli   ecoli   ecoli
/Users/charlesmurphy/Desktop/Research/data/genome/ecoli/bwa/ecoli.fa.amb
ecoli   ecoli   ecoli
/Users/charlesmurphy/Desktop/Research/data/genome/ecoli/bwa/ecoli.fa.ann
ecoli   ecoli   ecoli
/Users/charlesmurphy/Desktop/Research/data/genome/ecoli/bwa/ecoli.fa.bwt
ecoli   ecoli   ecoli
/Users/charlesmurphy/Desktop/Research/data/genome/ecoli/bwa/ecoli.fa.pac
ecoli   ecoli   ecoli
/Users/charlesmurphy/Desktop/Research/data/genome/ecoli/bwa/ecoli.fa.sa

I tried that and it does not work. Your response is appreciated.

Charlie


On Tue, Mar 5, 2013 at 7:11 PM, Ross  wrote:

> Hi, Charles,
>
> builds.txt is what tells the Galaxy framework about available genomes.
> Various binary index files created from those genomes are separately
> managed for specific tools since some folks won't need some application
> specific sets of indexes for any given genome - so you need to edit a
> different file!
>
> The BWA for illumina tool for example gets BWA index files from
> tool-data/bwa_index.loc so that's where the reference to your ecoli.fa and
> related BWA index files needs to be added - the format is described in the
> comments you'll find when you edit that file. Tabs for delimiters again and
> another restart should fix your problem for that tool?
>
>
>
>
>
>
> On Wed, Mar 6, 2013 at 11:46 AM, Charles Murphy  > wrote:
>
>> Hi,
>>
>> I' am trying to add built-in reference genomes for my local galaxy
>> installation. I know this topic has been covered multiple times: I have
>> looked through the discussion threads and the wiki (e.g. NGS Local Setup),
>> but I' am still unsuccessful. Previously I have just loaded a reference
>> genome through the history, but in the interest of saving time and space
>> while I experiment with Galaxy I wish to use the built-in feature.
>>
>> Here is what I have tried:
>>
>> 1. Indexed ecoli.fa using bwa and put all files (fasta and index files)
>> under /Users/charlesmurphy/Desktop/Research/data/genome/ecoli/bwa/
>>
>> The directory looks like:
>> -rwxrwxrwx@ 1 charlesmurphy  staff  4706043 Mar  5 14:54 ecoli.fa
>> -rwxrwxrwx  1 charlesmurphy  staff   12 Mar  5 16:38 ecoli.fa.amb
>> -rwxrwxrwx  1 charlesmurphy  staff  111 Mar  5 16:38 ecoli.fa.ann
>> -rwxrwxrwx  1 charlesmurphy  staff  4639752 Mar  5 16:38 ecoli.fa.bwt
>> -rwxrwxrwx  1 charlesmurphy  staff  1159920 Mar  5 16:38 ecoli.fa.pac
>> -rwxrwxrwx  1 charlesmurphy  staff  2319888 Mar  5 16:38 ecoli.fa.sa
>>
>> 2. Downloaded and installed the bwa wrapper files from the Galaxy main
>> toolshed, and put bwa in $PATH.
>> 3. Added the following line to the bwa_index.loc (using tabs as the
>> delimiter):
>> ecoli   ecoli   ecoli
>> /Users/charlesmurphy/Desktop/Research/data/genome/ecoli/bwa/ecoli.fa
>> 4. Added the following line to builds.txt (using tabs as the delimiter):
>> ecoli ecoli
>> 5. Restarted Galaxy.
>>
>> The ecoli genome does not show up as a reference genome for BWA (Map with
>> BWA for Illumina). Any help would be appreciated.
>>
>> Thanks,
>> Charlie
>>
>>
>>
___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:

  http://lists.bx.psu.edu/

[galaxy-dev] Tools and Data

[shenwiyn]

Hi,
I have some questions about the tool of Megablast in NGS:Mapping when I 
download the needed indexes .
There are four indexes files we need for Megablast:htgs 28-Jan-2013 ,nt 
28-Jan-2013,wgs 28-Jan-2013,phiX174;but there are too many types of the indexes 
file in ftp://ftp.ncbi.nlm.nih.gov/blast/db/ ,for example : 
nt.00.tar.gz,nt.01.tar.gz,nt.02.tar.gzI don't know the differents among 
these files ,and I don't which one  I should choose to download.In addition ,I 
can't find the index file for phiX174.
Could anyone help me and give me some advice please? Thank you very much.




shenwiyn___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:

  http://lists.bx.psu.edu/

Re: [galaxy-dev] Built-in reference genome

[Ross]

Hi, Charles,

builds.txt is what tells the Galaxy framework about available genomes.
Various binary index files created from those genomes are separately
managed for specific tools since some folks won't need some application
specific sets of indexes for any given genome - so you need to edit a
different file!

The BWA for illumina tool for example gets BWA index files from
tool-data/bwa_index.loc so that's where the reference to your ecoli.fa and
related BWA index files needs to be added - the format is described in the
comments you'll find when you edit that file. Tabs for delimiters again and
another restart should fix your problem for that tool?






On Wed, Mar 6, 2013 at 11:46 AM, Charles Murphy
wrote:

> Hi,
>
> I' am trying to add built-in reference genomes for my local galaxy
> installation. I know this topic has been covered multiple times: I have
> looked through the discussion threads and the wiki (e.g. NGS Local Setup),
> but I' am still unsuccessful. Previously I have just loaded a reference
> genome through the history, but in the interest of saving time and space
> while I experiment with Galaxy I wish to use the built-in feature.
>
> Here is what I have tried:
>
> 1. Indexed ecoli.fa using bwa and put all files (fasta and index files)
> under /Users/charlesmurphy/Desktop/Research/data/genome/ecoli/bwa/
>
> The directory looks like:
> -rwxrwxrwx@ 1 charlesmurphy  staff  4706043 Mar  5 14:54 ecoli.fa
> -rwxrwxrwx  1 charlesmurphy  staff   12 Mar  5 16:38 ecoli.fa.amb
> -rwxrwxrwx  1 charlesmurphy  staff  111 Mar  5 16:38 ecoli.fa.ann
> -rwxrwxrwx  1 charlesmurphy  staff  4639752 Mar  5 16:38 ecoli.fa.bwt
> -rwxrwxrwx  1 charlesmurphy  staff  1159920 Mar  5 16:38 ecoli.fa.pac
> -rwxrwxrwx  1 charlesmurphy  staff  2319888 Mar  5 16:38 ecoli.fa.sa
>
> 2. Downloaded and installed the bwa wrapper files from the Galaxy main
> toolshed, and put bwa in $PATH.
> 3. Added the following line to the bwa_index.loc (using tabs as the
> delimiter):
> ecoli   ecoli   ecoli
> /Users/charlesmurphy/Desktop/Research/data/genome/ecoli/bwa/ecoli.fa
> 4. Added the following line to builds.txt (using tabs as the delimiter):
> ecoli ecoli
> 5. Restarted Galaxy.
>
> The ecoli genome does not show up as a reference genome for BWA (Map with
> BWA for Illumina). Any help would be appreciated.
>
> Thanks,
> Charlie
>
>
>
___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:

  http://lists.bx.psu.edu/

[galaxy-dev] Built-in reference genome

[Charles Murphy]

Hi,

I' am trying to add built-in reference genomes for my local galaxy
installation. I know this topic has been covered multiple times: I have
looked through the discussion threads and the wiki (e.g. NGS Local Setup),
but I' am still unsuccessful. Previously I have just loaded a reference
genome through the history, but in the interest of saving time and space
while I experiment with Galaxy I wish to use the built-in feature.

Here is what I have tried:

1. Indexed ecoli.fa using bwa and put all files (fasta and index files)
under /Users/charlesmurphy/Desktop/Research/data/genome/ecoli/bwa/

The directory looks like:
-rwxrwxrwx@ 1 charlesmurphy  staff  4706043 Mar  5 14:54 ecoli.fa
-rwxrwxrwx  1 charlesmurphy  staff   12 Mar  5 16:38 ecoli.fa.amb
-rwxrwxrwx  1 charlesmurphy  staff  111 Mar  5 16:38 ecoli.fa.ann
-rwxrwxrwx  1 charlesmurphy  staff  4639752 Mar  5 16:38 ecoli.fa.bwt
-rwxrwxrwx  1 charlesmurphy  staff  1159920 Mar  5 16:38 ecoli.fa.pac
-rwxrwxrwx  1 charlesmurphy  staff  2319888 Mar  5 16:38 ecoli.fa.sa

2. Downloaded and installed the bwa wrapper files from the Galaxy main
toolshed, and put bwa in $PATH.
3. Added the following line to the bwa_index.loc (using tabs as the
delimiter):
ecoli   ecoli   ecoli
/Users/charlesmurphy/Desktop/Research/data/genome/ecoli/bwa/ecoli.fa
4. Added the following line to builds.txt (using tabs as the delimiter):
ecoli ecoli
5. Restarted Galaxy.

The ecoli genome does not show up as a reference genome for BWA (Map with
BWA for Illumina). Any help would be appreciated.

Thanks,
Charlie
___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:

  http://lists.bx.psu.edu/

Re: [galaxy-dev] How to count the number of Workflow steps?

[Dannon Baker]

Manually adding "HideDatasetActions" via the right panel in the editor was
deprecated a while back in favor of the workflow output 'asterisks' you see
rendered on each step in the workflow editor, but they're still used behind
the scenes.  When you flag a step as an output, all other non-output steps
should have HideDatasetActions added to the step automatically.

-Dannon



On Tue, Mar 5, 2013 at 5:32 PM, Richard Park  wrote:

> Hi everyone,
> Does anybody know how Galaxy currently determines which outputs from a
> tool are shown when a workflow is run? I'm currently dynamically creating
> workflows based on the number of given inputs and iterating through all
> "HideDatasetAction*" for each tools "post_job_actions" to figure out the
> total number of steps that will be run.
>
> "post_job_actions": {
> "HideDatasetActionoutput": {
> "action_arguments": {},
> "action_type": "HideDatasetAction",
> "output_name": "output"
> },
> "HideDatasetActionoutput_suppressed_reads_l": {
> "action_arguments": {},
> "action_type": "HideDatasetAction",
> "output_name": "output_suppressed_reads_l"
> },
> "HideDatasetActionoutput_suppressed_reads_r": {
> "action_arguments": {},
> "action_type": "HideDatasetAction",
> "output_name": "output_suppressed_reads_r"
> },
> "HideDatasetActionoutput_unmapped_reads_l": {
> "action_arguments": {},
> "action_type": "HideDatasetAction",
> "output_name": "output_unmapped_reads_l"
> },
> "HideDatasetActionoutput_unmapped_reads_r": {
> "action_arguments": {},
> "action_type": "HideDatasetAction",
> "output_name": "output_unmapped_reads_r"
> }
> },
>
> In the latest galaxy, there no longer seems to be
> any "HideDatasetAction*" in each tools "post_job_actions" options. How does
> Galaxy know which output files for a tool it should show or process?
>
> Does it have to do something with "RenameDatasetAction*"?
>
> "post_job_actions": {
> "RenameDatasetActionoutput_file": {
> "action_arguments": {
> "newname": "19_output_file"
> },
> "action_type": "RenameDatasetAction",
> "output_name": "output_file"
> }
> },
>
> Any guidance would be much appreciated.
> Thanks,
>
> ___
> Please keep all replies on the list by using "reply all"
> in your mail client.  To manage your subscriptions to this
> and other Galaxy lists, please use the interface at:
>
>   http://lists.bx.psu.edu/
>
___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:

  http://lists.bx.psu.edu/

[galaxy-dev] Workflow Actions

[Richard Park]

Hi everyone,
Does anybody know how Galaxy currently determines which outputs from a tool
are shown when a workflow is run? I'm currently dynamically creating
workflows based on the number of given inputs and iterating through all
"HideDatasetAction*" for each tools "post_job_actions" to figure out the
total number of steps that will be run.

"post_job_actions": {
"HideDatasetActionoutput": {
"action_arguments": {},
"action_type": "HideDatasetAction",
"output_name": "output"
},
"HideDatasetActionoutput_suppressed_reads_l": {
"action_arguments": {},
"action_type": "HideDatasetAction",
"output_name": "output_suppressed_reads_l"
},
"HideDatasetActionoutput_suppressed_reads_r": {
"action_arguments": {},
"action_type": "HideDatasetAction",
"output_name": "output_suppressed_reads_r"
},
"HideDatasetActionoutput_unmapped_reads_l": {
"action_arguments": {},
"action_type": "HideDatasetAction",
"output_name": "output_unmapped_reads_l"
},
"HideDatasetActionoutput_unmapped_reads_r": {
"action_arguments": {},
"action_type": "HideDatasetAction",
"output_name": "output_unmapped_reads_r"
}
},

In the latest galaxy, there no longer seems to be
any "HideDatasetAction*" in each tools "post_job_actions" options. How does
Galaxy know which output files for a tool it should show or process?

Does it have to do something with "RenameDatasetAction*"?

"post_job_actions": {
"RenameDatasetActionoutput_file": {
"action_arguments": {
"newname": "19_output_file"
},
"action_type": "RenameDatasetAction",
"output_name": "output_file"
}
},

Any guidance would be much appreciated.
Thanks,
Richard
___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:

  http://lists.bx.psu.edu/

[galaxy-dev] How to count the number of Workflow steps?

[Richard Park]

Hi everyone,
Does anybody know how Galaxy currently determines which outputs from a tool
are shown when a workflow is run? I'm currently dynamically creating
workflows based on the number of given inputs and iterating through all
"HideDatasetAction*" for each tools "post_job_actions" to figure out the
total number of steps that will be run.

"post_job_actions": {
"HideDatasetActionoutput": {
"action_arguments": {},
"action_type": "HideDatasetAction",
"output_name": "output"
},
"HideDatasetActionoutput_suppressed_reads_l": {
"action_arguments": {},
"action_type": "HideDatasetAction",
"output_name": "output_suppressed_reads_l"
},
"HideDatasetActionoutput_suppressed_reads_r": {
"action_arguments": {},
"action_type": "HideDatasetAction",
"output_name": "output_suppressed_reads_r"
},
"HideDatasetActionoutput_unmapped_reads_l": {
"action_arguments": {},
"action_type": "HideDatasetAction",
"output_name": "output_unmapped_reads_l"
},
"HideDatasetActionoutput_unmapped_reads_r": {
"action_arguments": {},
"action_type": "HideDatasetAction",
"output_name": "output_unmapped_reads_r"
}
},

In the latest galaxy, there no longer seems to be
any "HideDatasetAction*" in each tools "post_job_actions" options. How does
Galaxy know which output files for a tool it should show or process?

Does it have to do something with "RenameDatasetAction*"?

"post_job_actions": {
"RenameDatasetActionoutput_file": {
"action_arguments": {
"newname": "19_output_file"
},
"action_type": "RenameDatasetAction",
"output_name": "output_file"
}
},

Any guidance would be much appreciated.
Thanks,
___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:

  http://lists.bx.psu.edu/

Re: [galaxy-dev] Broken pipe in server

[James Taylor]

Broken pipe normally occurs when the browser drops the connection
(e.g. if you quit your browser while downloading a dataset you might
see this). Can you describe in more detail how the "frequent hiccups"
manifest?

--
James Taylor, Assistant Professor, Biology/CS, Emory University


On Mon, Mar 4, 2013 at 12:36 PM, Thomas van Gurp
 wrote:
> Dear all,
>
> Our galaxy suffers from frequent hickups, which apparently are due to broken
> pipes.
>
> It will give these errors in paster.log
>
> 
> Exception happened during processing of request from ('127.0.0.1', 53234)
> Traceback (most recent call last):
>   File
> "/usr/local/lib/python2.7/dist-packages/Paste-1.7.5.1-py2.7.egg/paste/httpserver.py",
> line 1068, in process_request_in_thread
> self.finish_request(request, client_address)
>   File "/usr/lib/python2.7/SocketServer.py", line 323, in finish_request
> self.RequestHandlerClass(request, client_address, self)
>   File "/usr/lib/python2.7/SocketServer.py", line 641, in __init__
> self.finish()
>   File "/usr/lib/python2.7/SocketServer.py", line 694, in finish
> self.wfile.flush()
>   File "/usr/lib/python2.7/socket.py", line 303, in flush
> self._sock.sendall(view[write_offset:write_offset+buffer_size])
> error: [Errno 32] Broken pipe
> 
>
>
> Any idea why this might be an what I would need to check?
>
> Cheers,
> THomas
>
> ___
> Please keep all replies on the list by using "reply all"
> in your mail client.  To manage your subscriptions to this
> and other Galaxy lists, please use the interface at:
>
>   http://lists.bx.psu.edu/
___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:

  http://lists.bx.psu.edu/

Re: [galaxy-dev] errors with run_functional_tests.sh

[Dave Bouvier]

Hello,

I'll respond inline, but your base understanding is correct.

   --Dave B.

On 3/5/13 12:17:47.000, Florence Maurier wrote:

Hi,

We are currently installing galaxy on a server. By reading the galaxy
wiki, we found the script run_functional_tests.sh (the principle is
really handy!). Unfortunately, we encounter errors when we use it. I
hope someone can help us.

In reality, we encounter several types of errors ...
The first is present in each run, for example for the execution follows:

./run_functional_tests.sh -id fastqc
...
FastQC:Read QC ( fastqc ) > Test-1 ... galaxy.web.framework DEBUG
2013-03-01 09:42:10,466
...
galaxy.tools.actions.upload_common INFO 2013-03-01 09:42:13,357 tool
upload1 created job id 1
...
galaxy.jobs DEBUG 2013-03-01 09:42:27,436 Tool did not define exit code
or stdio handling; checking stderr for success
galaxy.jobs DEBUG 2013-03-01 09:42:27,757 job 1 ended

galaxy.tools.actions.upload_common INFO 2013-03-01 09:42:40,883 tool
upload1 created job id 2
...
galaxy.jobs DEBUG 2013-03-01 09:42:52,000 Tool did not define exit code
or stdio handling; checking stderr for success
galaxy.jobs DEBUG 2013-03-01 09:42:52,311 job 2 ended

galaxy.jobs.manager DEBUG 2013-03-01 09:42:55,285 (3) Job assigned to
handler 'main'
galaxy.jobs DEBUG 2013-03-01 09:43:00,477 (3) Working directory for job
is: /var/local/galaxy/galaxy-dist/database/job_working_directory/000/3
galaxy.jobs.handler DEBUG 2013-03-01 09:43:00,478 dispatching job 3 to
local runner
galaxy.jobs.handler INFO 2013-03-01 09:43:00,622 (3) Job dispatched
galaxy.jobs.runners.local DEBUG 2013-03-01 09:43:00,727 Local runner:
starting job 3
galaxy.jobs.runners.local DEBUG 2013-03-01 09:43:01,193 executing:
python /var/local/galaxy/galaxy-dist/tools/rgenetics/rgFastQC.py -i
"/tmp/tmp_YcL4J/database/files/000/dataset_1.dat" -d
"/var/local/galaxy/galaxy-dist/database/job_working_directory/000/3/dataset_3_files"
-o "/tmp/tmp_YcL4J/database/files/000/dataset_3.dat" -n "fastqcout" -f
"fastq" -j "1000gsample.fastq" -e
"/var/local/galaxy/galaxy-dist/tool-data/shared/jars/FastQC/fastqc" -c
"/tmp/tmp_YcL4J/database/files/000/dataset_2.dat"
galaxy.jobs.runners.local DEBUG 2013-03-01 09:43:04,515 execution
finished: python
/var/local/galaxy/galaxy-dist/tools/rgenetics/rgFastQC.py -i
"/tmp/tmp_YcL4J/database/files/000/dataset_1.dat" -d
"/var/local/galaxy/galaxy-dist/database/job_working_directory/000/3/dataset_3_files"
-o "/tmp/tmp_YcL4J/database/files/000/dataset_3.dat" -n "fastqcout" -f
"fastq" -j "1000gsample.fastq" -e
"/var/local/galaxy/galaxy-dist/tool-data/shared/jars/FastQC/fastqc" -c
"/tmp/tmp_YcL4J/database/files/000/dataset_2.dat"
galaxy.jobs DEBUG 2013-03-01 09:43:04,578 Tool did not define exit code
or stdio handling; checking stderr for success
_galaxy.tools DEBUG 2013-03-01 09:43:04,814 Error opening galaxy.json
file: [Errno 2] No such file or directory:
'/var/local/galaxy/galaxy-dist/database/job_working_directory/000/3/galaxy.json'_

galaxy.jobs DEBUG 2013-03-01 09:43:04,899 job 3 ended
ok
--
Ran 1 test in 60.889s
OK
...
Exception OSError: (2, 'No such file or directory',
'/tmp/tmp_YcL4J/database/tmp/tmpn5w0Ln') in ', mode 'w' at
0x18ffd5d0>> ignored


According to other messages from the mailing list, my understanding is
that this message should be ignored and we only have to consider the
"OK" that says the test went well. Is that correct ?


Yes, the error messages you see there are irrelevant, and can be safely 
ignored. As you deduced, the functional test passed.




The other type of error we find is the following:
./run_functional_tests.sh -id cshl_fastx_clipper
...
ERROR: Failure: ValueError (No such test TestForTool_cshl_fastx_clipper)
--
Traceback (most recent call last):
   File
"/var/local/galaxy/galaxy-dist/eggs/nose-0.11.1-py2.6.egg/nose/failure.py",
line 39, in runTest
 raise self.exc_class(self.exc_val)
ValueError: No such test TestForTool_cshl_fastx_clipper
--
...
For some of these tools, the error seems logical given that the xml
configuration tool does not contain test. However, for others like
fastx_clipper, the test is described... So, why is there a problem with
this kind of tools ?


In the case of fastx_clipper, the test definition is commented out, and 
therefore not found in the toolbox.




Finally, we have one last type of error, like this:
./run_functional_tests.sh -id sam_to_bam
...
SAM-to-BAM ( sam_to_bam ) > Test-2 ... galaxy.web.framework DEBUG
2013-03-05 18:02:15,115 Error: this request returned None from
get_history(): http://localhost:9051/user/logout
galaxy.web.framework DEBUG 2013-03-05 18:02:15,159 Error: this request
returned None from get_history(): http://localhost:9051/
ERROR

==
ERROR: SAM-to-BAM ( sam_to_bam ) > T

Re: [galaxy-dev] Authentication: email addresses are not validated prior to account creation

[James Taylor]

repoze.who would seem like the best candidate these days, it would be great
to see that integrated, but I worry it would also cause lots of
unintentional breakage in the corner cases.
On Mar 5, 2013 12:27 PM, "Paul Boddie"  wrote:

> On 05/03/13 17:09, James Taylor wrote:
>
>> On Mar 1, 2013, at 10:39 AM, Vipin TS  wrote:
>>
>>  Hello members,
>>>
>>> I believe currently there is no process to validate email address
>>> provided during user account creation. We are experiencing a huge fake
>>> account creation attack on our public facing galaxy instance.
>>>
>>> Does anybody who has been managing a public instance, implemented an
>>> on-demand account creation activation by sending an email containing a
>>> link, which when clicked, validate the account creation request. Or any
>>> plans from dev-team to add this in future release?
>>>
>> How about some kind of captcha support?
>>
>
> Recently, there has been increased awareness of some of the pitfalls
> involved in managing identity and authentication-related information in
> Python-based applications - not specifically to do with Python itself, but
> more to do with the community and the perceived best practices - and I'd
> really like to see a bit more collaboration around those things as well as
> around anti-spam mechanisms. Having looked at the authentication aspects of
> Galaxy, I can't help wondering if there shouldn't be some kind of generic
> "shell" for such functionality that is separate from the core functionality
> of Galaxy and would be used for other systems as well. Certainly, using
> Apache is one solution, but people do seem to want a more controlled kind
> of integration between that and the underlying applications.
>
> At the very least, one would hope to reuse and integrate existing
> components, perhaps at the WSGI level. Failing that, there might be some
> generic libraries that could support such reusable components. Perhaps the
> most significant challenge would be to cleanly integrate the user interface
> aspects of such components with the Galaxy output.
>
> Certainly, one could just extend the registration mechanism with captcha
> support, but I'd be worried about the maintainability of the code. Unless
> things have progressed fairly recently, there was already a lot of
> special-cased stuff in the area of authentication, and I'd be worried about
> unintentional breakage.
>
> Paul
>
___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:

  http://lists.bx.psu.edu/

Re: [galaxy-dev] Authentication: email addresses are not validated prior to account creation

[Paul Boddie]

On 05/03/13 17:09, James Taylor wrote:

On Mar 1, 2013, at 10:39 AM, Vipin TS  wrote:


Hello members,

I believe currently there is no process to validate email address provided 
during user account creation. We are experiencing a huge fake account creation 
attack on our public facing galaxy instance.

Does anybody who has been managing a public instance, implemented an on-demand 
account creation activation by sending an email containing a link, which when 
clicked, validate the account creation request. Or any plans from dev-team to 
add this in future release?

How about some kind of captcha support?


Recently, there has been increased awareness of some of the pitfalls 
involved in managing identity and authentication-related information in 
Python-based applications - not specifically to do with Python itself, 
but more to do with the community and the perceived best practices - and 
I'd really like to see a bit more collaboration around those things as 
well as around anti-spam mechanisms. Having looked at the authentication 
aspects of Galaxy, I can't help wondering if there shouldn't be some 
kind of generic "shell" for such functionality that is separate from the 
core functionality of Galaxy and would be used for other systems as 
well. Certainly, using Apache is one solution, but people do seem to 
want a more controlled kind of integration between that and the 
underlying applications.


At the very least, one would hope to reuse and integrate existing 
components, perhaps at the WSGI level. Failing that, there might be some 
generic libraries that could support such reusable components. Perhaps 
the most significant challenge would be to cleanly integrate the user 
interface aspects of such components with the Galaxy output.


Certainly, one could just extend the registration mechanism with captcha 
support, but I'd be worried about the maintainability of the code. 
Unless things have progressed fairly recently, there was already a lot 
of special-cased stuff in the area of authentication, and I'd be worried 
about unintentional breakage.


Paul
___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:

 http://lists.bx.psu.edu/

[galaxy-dev] errors with run_functional_tests.sh

[Florence Maurier]

Hi,

We are currently installing galaxy on a server. By reading the galaxy 
wiki, we found the script run_functional_tests.sh (the principle is 
really handy!). Unfortunately, we encounter errors when we use it. I 
hope someone can help us.


In reality, we encounter several types of errors ...
The first is present in each run, for example for the execution follows:

./run_functional_tests.sh -id fastqc
...
FastQC:Read QC ( fastqc ) > Test-1 ... galaxy.web.framework DEBUG 
2013-03-01 09:42:10,466

...
galaxy.tools.actions.upload_common INFO 2013-03-01 09:42:13,357 tool 
upload1 created job id 1

...
galaxy.jobs DEBUG 2013-03-01 09:42:27,436 Tool did not define exit code 
or stdio handling; checking stderr for success

galaxy.jobs DEBUG 2013-03-01 09:42:27,757 job 1 ended

galaxy.tools.actions.upload_common INFO 2013-03-01 09:42:40,883 tool 
upload1 created job id 2

...
galaxy.jobs DEBUG 2013-03-01 09:42:52,000 Tool did not define exit code 
or stdio handling; checking stderr for success

galaxy.jobs DEBUG 2013-03-01 09:42:52,311 job 2 ended

galaxy.jobs.manager DEBUG 2013-03-01 09:42:55,285 (3) Job assigned to 
handler 'main'
galaxy.jobs DEBUG 2013-03-01 09:43:00,477 (3) Working directory for job 
is: /var/local/galaxy/galaxy-dist/database/job_working_directory/000/3
galaxy.jobs.handler DEBUG 2013-03-01 09:43:00,478 dispatching job 3 to 
local runner

galaxy.jobs.handler INFO 2013-03-01 09:43:00,622 (3) Job dispatched
galaxy.jobs.runners.local DEBUG 2013-03-01 09:43:00,727 Local runner: 
starting job 3
galaxy.jobs.runners.local DEBUG 2013-03-01 09:43:01,193 executing: 
python /var/local/galaxy/galaxy-dist/tools/rgenetics/rgFastQC.py -i 
"/tmp/tmp_YcL4J/database/files/000/dataset_1.dat" -d 
"/var/local/galaxy/galaxy-dist/database/job_working_directory/000/3/dataset_3_files" 
-o "/tmp/tmp_YcL4J/database/files/000/dataset_3.dat" -n "fastqcout" -f 
"fastq" -j "1000gsample.fastq" -e 
"/var/local/galaxy/galaxy-dist/tool-data/shared/jars/FastQC/fastqc" -c 
"/tmp/tmp_YcL4J/database/files/000/dataset_2.dat"
galaxy.jobs.runners.local DEBUG 2013-03-01 09:43:04,515 execution 
finished: python 
/var/local/galaxy/galaxy-dist/tools/rgenetics/rgFastQC.py -i 
"/tmp/tmp_YcL4J/database/files/000/dataset_1.dat" -d 
"/var/local/galaxy/galaxy-dist/database/job_working_directory/000/3/dataset_3_files" 
-o "/tmp/tmp_YcL4J/database/files/000/dataset_3.dat" -n "fastqcout" -f 
"fastq" -j "1000gsample.fastq" -e 
"/var/local/galaxy/galaxy-dist/tool-data/shared/jars/FastQC/fastqc" -c 
"/tmp/tmp_YcL4J/database/files/000/dataset_2.dat"
galaxy.jobs DEBUG 2013-03-01 09:43:04,578 Tool did not define exit code 
or stdio handling; checking stderr for success
_galaxy.tools DEBUG 2013-03-01 09:43:04,814 Error opening galaxy.json 
file: [Errno 2] No such file or directory: 
'/var/local/galaxy/galaxy-dist/database/job_working_directory/000/3/galaxy.json'_

galaxy.jobs DEBUG 2013-03-01 09:43:04,899 job 3 ended
ok
--
Ran 1 test in 60.889s
OK
...
Exception OSError: (2, 'No such file or directory', 
'/tmp/tmp_YcL4J/database/tmp/tmpn5w0Ln') in _TemporaryFileWrapper.__del__ of ', mode 'w' at 
0x18ffd5d0>> ignored



According to other messages from the mailing list, my understanding is 
that this message should be ignored and we only have to consider the 
"OK" that says the test went well. Is that correct ?


The other type of error we find is the following:
./run_functional_tests.sh -id cshl_fastx_clipper
...
ERROR: Failure: ValueError (No such test TestForTool_cshl_fastx_clipper)
--
Traceback (most recent call last):
  File 
"/var/local/galaxy/galaxy-dist/eggs/nose-0.11.1-py2.6.egg/nose/failure.py", 
line 39, in runTest

raise self.exc_class(self.exc_val)
ValueError: No such test TestForTool_cshl_fastx_clipper
--
...
For some of these tools, the error seems logical given that the xml 
configuration tool does not contain test. However, for others like 
fastx_clipper, the test is described... So, why is there a problem with 
this kind of tools ?


Finally, we have one last type of error, like this:
./run_functional_tests.sh -id sam_to_bam
...
SAM-to-BAM ( sam_to_bam ) > Test-2 ... galaxy.web.framework DEBUG 
2013-03-05 18:02:15,115 Error: this request returned None from 
get_history(): http://localhost:9051/user/logout
galaxy.web.framework DEBUG 2013-03-05 18:02:15,159 Error: this request 
returned None from get_history(): http://localhost:9051/

ERROR

==
ERROR: SAM-to-BAM ( sam_to_bam ) > Test-2
--
Traceback (most recent call last):
  File "/var/local/galaxy/galaxy-dist/test/functional/test_toolbox.py", 
line 171, in test_tool

self.do_it( td, shed_tool_id=shed_tool_id )
  File "/var/local/galaxy/galaxy-dis

[galaxy-dev] NCBI rpsblast and rpstblastn for CDD searches etc

[Peter Cock]

On Mon, Feb 4, 2013 at 6:14 PM, Peter Cock  wrote:
> On Mon, Feb 4, 2013 at 5:43 PM, Luobin Yang  wrote:
>> Hi, Peter,
>>
>> What difficulties did you see to include psi-blast and phi-blast when you
>> were working on the other programs in the NCBI+ suite? I would be
>> interested in creating a tool configuration file for each of these two.
>>
>> Thanks,
>> Luobin
>
> Hi Luobin,
>
> At first glance, the binaries for psiblast, rpsblast, and rpstblastn don't
> look too different for wrapping. We would need to define a third loc
> file for the RPS BLAST domain databases (like the existing loc files
> for nucleotide and protein databases).

I've got wrappers working for rpsblast and rpstblastn with a new
blastdb_d.loc file for the associated domain databases, currently
in my bitbucket repository and live on our internal Galaxy server.
Barring any unexpected problems, I'll include this in an update
to the Tool Shed in a week or so - so watch this space:
http://toolshed.g2.bx.psu.edu/view/devteam/ncbi_blast_plus/

Meanwhile work on wrapping psiblast is ongoing, with several
contributors. We'll need to have ASN.1 base formats defined
in Galaxy first (discussed on another thread).

Regards,

Peter
___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:

  http://lists.bx.psu.edu/

Re: [galaxy-dev] Authentication: email addresses are not validated prior to account creation

[James Taylor]

How about some kind of captcha support?

-- jt 

(composed on my phone)

On Mar 1, 2013, at 10:39 AM, Vipin TS  wrote:

> Hello members, 
> 
> I believe currently there is no process to validate email address provided 
> during user account creation. We are experiencing a huge fake account 
> creation attack on our public facing galaxy instance. 
> 
> Does anybody who has been managing a public instance, implemented an 
> on-demand account creation activation by sending an email containing a link, 
> which when clicked, validate the account creation request. Or any plans from 
> dev-team to add this in future release? 
> 
> thanks in advance, 
> --/Vipin
> ___
> Please keep all replies on the list by using "reply all"
> in your mail client.  To manage your subscriptions to this
> and other Galaxy lists, please use the interface at:
> 
>  http://lists.bx.psu.edu/
___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:

  http://lists.bx.psu.edu/

Re: [galaxy-dev] Authentication: email addresses are not validated prior to account creation

[Dannon Baker]

Vipin,

I don't know of anyone who's done this, but if anyone wants to take a stab
at it, it's definitely something that we could include (as long as it's
optional, configurable in universe_wsgi.ini) in Galaxy.

-Dannon

On Fri, Mar 1, 2013 at 10:39 AM, Vipin TS  wrote:

> Hello members,
>
> I believe currently there is no process to validate email address provided
> during user account creation. We are experiencing a huge fake account
> creation attack on our public facing galaxy 
> instance
> .
>
> Does anybody who has been managing a public instance, implemented an
> on-demand account creation activation by sending an email containing a
> link, which when clicked, validate the account creation request. Or any
> plans from dev-team to add this in future release?
>
> thanks in advance,
> --/Vipin
>
> ___
> Please keep all replies on the list by using "reply all"
> in your mail client.  To manage your subscriptions to this
> and other Galaxy lists, please use the interface at:
>
>   http://lists.bx.psu.edu/
>
___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:

  http://lists.bx.psu.edu/

Re: [galaxy-dev] Galaxy: Dependency error htseq_count

[Lance Parsons]

Hi Joachim,

It seems that the error is because your system is trying to install the 
library files to lib64/python instead of just lib/python.  For reasons I 
don't quite understand, the setup.py installation fails when that 
directory doesn't exist (I would think it would just create it, but that 
doesn't appear to be the case).  As a quick fix, you might try creating 
the directory 
/apps/htseq/0.5.3p9/lparsons/htseq_count/f320093f1e8e/lib64/python/.  
Create it as the user that galaxy runs under so make sure galaxy has 
permissions later.  Then try reinstalling the dependency.  I'm not sure 
that will work, but that is the gist of the problem.


I noticed there is an update to the htseq-count package, so I'll try to 
put out a new version of the wrapper with a fix for that as well as the 
new version.  Let me know if you still have trouble or if you find some 
sort of workaround or fix.  I'd like to make the installation more robust.


Lance

Joachim Jacob | VIB | wrote:

Hi Lance,

[apologies for directly emailing to you and not through the galaxy-dev 
list. I will post it on the dev list later on]


I am trying to install htseq-count, but it fails during the dependency 
installation of htseq.


The error I get is:

error: can't create or remove files in install directory

The following error occurred while trying to add or remove files in the
installation directory:

[Errno 2] No such file or directory: 
'/apps/htseq/0.5.3p9/lparsons/htseq_count/f320093f1e8e/lib64/python/test-easy-install-30482.pth' 



The installation directory you specified (via --install-dir, --prefix, or
the distutils default setting) was:

/apps/htseq/0.5.3p9/lparsons/htseq_count/f320093f1e8e/lib64/python/

This directory does not currently exist. Please create it and try 
again, or

choose a different installation directory (using the -d or --install-dir
option).

Any idea whether it is related to my setup (and what I can do about 
it)? Note: only the htseq_count dependency is not being installed.
Thanks for providing the htseq_count wrapper! Now getting it through 
the toolshed will make it again a little closer to a perfect world :-)



Cheers,
Joachim

Joachim Jacob

Rijvisschestraat 120, 9052 Zwijnaarde
Tel: +32 9 244.66.34
Bioinformatics Training and Services (BITS)
http://www.bits.vib.be
@bitsatvib

On 12/17/2012 05:28 PM, Lance Parsons wrote:
Thanks for looking into this Greg.  The environment where I'm having 
this problem is our testing and production systems.


Both environments are CentOS release 5.8 (Final) with Mercurial 
version 2.4.1.


Interestingly, when I tried the same thing with a development setup 
on my OSX box (10.6.8) with Mercurial 2.4 I did not have the same 
issue. One difference is that on my testing/production boxes I do 
have an older version of the tool installed, so perhaps that is the 
issue.


I also tried uninstalling, but that led to the (expected) error that 
that a previous changeset is installed:
The tool shed repository htseq_count with owner lparsons and 
changeset revision f320093f1e8e was previously installed using 
changeset revision 5d969cb56112. The repository has been uninstalled, 
however, so reinstall the original repository instead of installing 
it again. You can get the latest updates for the repository using the 
Get updates option from the repository's Repository Actions pop-up 
menu. Click here to manage the repository.


I then attempted to reinstall and I got a javascript message the 
cloning failed, and the repo is now "stuck" in the cloning state.  I 
did not see anything in the paster logs regarding this at all.



Here are the steps I took on my local system:

  1. Upload htseq-count v0.3 to local toolshed

  2. Install on local galaxy

  3. Upload htseq-count v0.3-release2 to toolshed

[Errno 2] No such file or directory: 
'/Users/lparsons/Documents/projects/sequencing/galaxy/galaxy-test1/database/community_files/000/repo_3/./tool_data_table_conf.xml.sample' 



STDOUT from paster
 htseq-count.xml does not exist!
sam_fa_indices.loc.sample does not exist!
static/images/count_modes.png does not exist!
test-data/htseq-test-paired.bam does not exist!
test-data/htseq-test-paired_counts.tsv does not exist!
test-data/htseq-test-paired_othercounts.tsv does not exist!
test-data/htseq-test.bam does not exist!
test-data/htseq-test.gff does not exist!
test-data/htseq-test.sam does not exist!
test-data/htseq-test_counts.tsv does not exist!
test-data/htseq-test_othercounts.tsv does not exist!
tool_data_table_conf.xml.sample does not exist!
  4. Second attempt to upload
The file 'htseq-count_0.3.release2.tar.gz' has been successfully 
uncompressed and uploaded to the repository. 1 undesirable items (.hg 
.svn .git directories, .DS_Store, hgrc files, etc) were removed from 
the archive. 13 files were removed from the repository root.


  5. Get repository updates
The installed repository named 'htseq_count' has been updated to 
change set revision '74a6a8221337'.




G

Re: [galaxy-dev] Tools and Data

[shenwiyn]

Hi Enis Afgan,
Thank you very much for you answer.
I have some questions abount the tool of Megablast in NGS:Mapping when I 
download the needed indexes .
There are four indexes files we need for Megablast:htgs 28-Jan-2013 ,nt 
28-Jan-2013,wgs 28-Jan-2013,phiX174;but there are too many types of the indexes 
file in ftp://ftp.ncbi.nlm.nih.gov/blast/db/ ,for example : 
nt.00.tar.gz,nt.01.tar.gz,nt.02.tar.gzI don't know the differents among 
these files ,and I don't which one  I should choose to download.In addition ,I 
can't find the index file for phiX174.
Could you help me and give me some advice please? Thanks.




shenwiyn

From: Enis Afgan
Date: 2013-03-05 10:38
To: shenwiyn
CC: Rafael Hernández; galaxy-...@bx.psu.edu
Subject: Re: Tools and Data
Correct; certain NSG tools require additional configuration for their indices 
(http://wiki.galaxyproject.org/Admin/NGS%20Local%20Setup)



On Tue, Mar 5, 2013 at 1:07 PM, shenwiyn  wrote:

Hi,
I've installed Galaxy in local as what Rafael does.I want to make sure that the 
basic functions work successfully just depending on the necessary tools in Tool 
dependencies manual,and the NGS tools not only depend on the necessary tools 
but also some another needed data(such as indexes,and so on).

thakks



shenwy___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:

  http://lists.bx.psu.edu/

Re: [galaxy-dev] [galaxy-iuc] Installing repositories in a specific order

[Ira Cooke]

Hi Peter, 

The problem is that I have multiple repositories in my repository_dependency 
file ... but they need to be installed in a specific order.  I can't seem to 
get galaxy to install them in the correct order.

Ira

PS: Sorry about the totally irrelevant subject line ... I've changed it.

On 05/03/2013, at 9:01 PM, Peter Cock  wrote:

> On Tue, Mar 5, 2013 at 6:49 AM, Ira Cooke  wrote:
>> Hi,
>> 
>> I've just uploaded a set of updates to my proteomics tools on the toolshed.
>> These contain my best attempt to implement automatic dependency installation
>> ... and for the most part it seems to work.
>> 
>> I have encountered the following issues though
>> 
>> (1) I don't see any way to ensure that one repository_dependency is
>> installed before all others.  In my case this is important because I have an
>> installer package (galaxy_protk) which needs to be installed before other
>> package installs will work.
>> 
>> For example .. try installing
>> 
>> http://toolshed.g2.bx.psu.edu/view/iracooke/omssa
> 
> I think in this case you just need to use a repository_dependencies.xml
> file (but note currently this only lets you point at other repositories in
> the same Tool Shed instance, i.e. http://toolshed.g2.bx.psu.edu/ here):
> http://wiki.galaxyproject.org/DefiningRepositoryDependencies
> 
> Peter


___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:

  http://lists.bx.psu.edu/

[galaxy-dev] Proteomics category on test toolshed

[Ira Cooke]

Hi, 

I've just uploaded a set of updates to my proteomics tools on the toolshed.  
These contain my best attempt to implement automatic dependency installation 
... and for the most part it seems to work. 

I have encountered the following issues though

(1) I don't see any way to ensure that one repository_dependency is installed 
before all others.  In my case this is important because I have an installer 
package (galaxy_protk) which needs to be installed before other package 
installs will work.

For example .. try installing

http://toolshed.g2.bx.psu.edu/view/iracooke/omssa

(2) Running multiple repo installs at once is not reliable (sorry this is so 
vague).  When I have many dependencies being installed at once I get the 
following error (for example)

"Could not cd to 
./database/tmp/tmpbIqBW8/galaxy_protk/./database/tmp/tmp3vhKn6/galaxy_protk"

It looks like a path is not being constructed correctly. 

If I wait until the currently installed repo is finished .. then try 
uninstalling and reinstalling it works.
Sorry this is vague but if you try installing any of the following repositories 
you should see what I mean

http://toolshed.g2.bx.psu.edu/view/iracooke/xtandem
http://toolshed.g2.bx.psu.edu/view/iracooke/msgfplus
http://toolshed.g2.bx.psu.edu/view/iracooke/tpp_prophets

Anyway .. and just so it doesn't just sound like I'm complaining ... I think 
the dependency installation stuff is hugely useful.  It took me a while to 
figure out all the details .. but I think it makes sense now.

Cheers
Ira

___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:

  http://lists.bx.psu.edu/

[galaxy-dev] GalaxyAdmins meetup on March 20th at 10 AM Central

[IIHG Galaxy Administrator]

Hello,

The next GalaxyAdmins Meetup will be held on Wednesday, March 20, 2013 from 10 
AM to 11 AM US Central time at 
https://globalcampus.uiowa.edu:443/join_meeting.html?meetingId=1262344508128.  
Hailiang (Leon) Mei and David van Enckevort will present on "NBIC Galaxy at 
SURFsara's HPC cloud".  There will also be a Galaxy Project Update by a Galaxy 
Team member.

We encourage and look forward to your participation in the meetup.

Downloading the required Blackboard Collaborate web conferencing system applet 
in advance, and using a headphone with a microphone to prevent audio feedback 
during the call is recommended.

Thank you.

--
Srinivas Maddhi
Senior Systems Administrator
Iowa Institute for Human Genetics
ITS Research Services, The University of Iowa
BEGIN:VCALENDAR
PRODID:-//Blackboard Collaborate//NONSGML v1.0//EN
VERSION:2.0
METHOD:PUBLISH
BEGIN:VEVENT
DTSTART:20130320T15Z
DTEND:20130320T16Z
LOCATION:Blackboard Collaborate Virtual Room
TRANSP:OPAQUE
DTSTAMP:20130304T203540Z
SUMMARY:NBIC Galaxy at SURFsara's HPC cloud
DESCRIPTION:You have been invited to attend the following online Blackboard
  Collaborate Web Conference session created by smaddhi (Maddhi\,
  Srinivas).\nYou have been assigned the Moderator role.\nSession Name:\n  
   NBIC Galaxy at SURFsara's HPC cloud\n\nSession Details:\nStarts: Mar
  20\, 2013 10:00 AM\nEnds: Mar 20\, 2013 11:00 AM\nJoin Session:
  https://globalcampus.uiowa.edu:443/join_meeting.html?meetingId=12623445081
 28\n\n\n\nTo join the session\, click on the Join Session link.\nEnter
  your Elluminate Live! Manager login name and password.\n\nYou may join
  the session 60 minutes prior to the start time.\nIf this is the first
  time you will be using Elluminate\, you may be prompted to download some
  software which may take anywhere from 2 to 20 minutes depending upon your
  Internet connection speed. You can pre-configure your system with the
  required software by going to the support page located at:
  http://support.blackboardcollaborate.com/\nPlease make sure your computer
  has a microphone and speakers to be able to talk and hear while you are
  in the Blackboard Collaborate Web Conference session.

PRIORITY:3
UID:20130320T15Z20130320T16Z
CLASS:PUBLIC
END:VEVENT
END:VCALENDAR___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:

  http://lists.bx.psu.edu/

Re: [galaxy-dev] [galaxy-user] Setting up galaxy local server

[Nate Coraor]

On Mar 3, 2013, at 5:21 PM, Anusha Nagari wrote:

> Hi,
> 
> I am a new user of galaxy and am trying to set up local version of galaxy on 
> my mac. I followed the instructions in this link and successfully installed 
> the local version using the following steps:
> 
> mkdir galaxy-python/
> mkdir galaxy-dist/
> ln -s /usr/local/bin/ galaxy-python/python
> export PATH=~/galaxy-python:$PATH
> hg clone https://bitbucket.org/galaxy/galaxy-dist/
> cd galaxy-dist
> sh run.sh
> 
> I could load galaxy fine for the first time. However after I closed it and 
> tried to run it by sh run.sh, it is giving the following error and am unable 
> to startup galaxy:
> __import__( "pkg_resources" ).declare_namespace( __name__ )
> Traceback (most recent call last):
> File "./scripts/manage_tools.py", line 12, in 
> from migrate.versioning.shell import main
> ImportError: No module named migrate.versioning.shell

Hi Anusha,

I've moved this thread over to the galaxy-dev list since it concerns a local 
installation. Could you check to see if you've set the PYTHONPATH environment 
variable?  It seems that something odd is going on with your environment.  
Also, if you opened a new shell for the second time you executed run.sh, you'll 
need to set the same PATH variable as you did the first time.  That export does 
not persist unless you place it in one of your shell's startup files.

The symlink method is quick and easy, but if you are going to have a persistent 
Galaxy server that you'll be installing python modules in to, I would suggest 
using a virtualenv as described here:

http://wiki.galaxyproject.org/Admin/Config/Performance/ProductionServer#Use_a_clean_environment

--nate

> 
> However, I am not able to upload any datasets into galaxy either from UCSC 
> genome browser or from my computer with the following error:
> An error occurred with this dataset: 
> /Users/anushanagari/Desktop/Softwares/galaxy-dist/lib/galaxy/__init__.py:5: 
> UserWarning: Module site was already imported from 
> /Library/Frameworks/Python.framework/Versions/4.2.30201/lib/python2.5/site.pyc,
>  but /Library/Frameworks/Python.framework/Version
> 
> Tool execution generated the following error message:
> /Users/anushanagari/Desktop/Softwares/galaxy-dist/lib/galaxy/__init__.py:5: 
> UserWarning: Module site was already imported from 
> /Library/Frameworks/Python.framework/Versions/4.2.30201/lib/python2.5/site.pyc,
>  but 
> /Library/Frameworks/Python.framework/Versions/4.2.30201/lib/python2.5/site-packages/Enstaller-3.0.9-py2.5.egg
>  is being added to sys.path
>   __import__( "pkg_resources" ).declare_namespace( __name__ )
> /Users/anushanagari/Desktop/Softwares/galaxy-dist/lib/galaxy/__init__.py:5: 
> UserWarning: Module pkg_resources was already imported from 
> /Users/anushanagari/Desktop/Softwares/galaxy-dist/lib/pkg_resources.pyc, but 
> /Library/Frameworks/Python.framework/Versions/4.2.30201/lib/python2.5/site-packages/Enstaller-3.0.9-py2.5.egg
>  is being added to sys.path
>   __import__( "pkg_resources" ).declare_namespace( __name__ )
> 
> 
> Could you please help me with this and also provide any additional 
> documentation that could help me in getting to know galaxy better? Thanks in 
> advance! Hoping to hear from you soon!
> 
> Best,
> Anusha
> ___
> The Galaxy User list should be used for the discussion of
> Galaxy analysis and other features on the public server
> at usegalaxy.org.  Please keep all replies on the list by
> using "reply all" in your mail client.  For discussion of
> local Galaxy instances and the Galaxy source code, please
> use the Galaxy Development list:
> 
>  http://lists.bx.psu.edu/listinfo/galaxy-dev
> 
> To manage your subscriptions to this and other Galaxy lists,
> please use the interface at:
> 
>  http://lists.bx.psu.edu/


___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:

  http://lists.bx.psu.edu/

Re: [galaxy-dev] [galaxy-iuc] Proteomics category on test toolshed

[Greg Von Kuster]

Hello Ira,

Thanks for all of the work you've been doing n your repositories - I've been 
getting email updates from your repository changes.

I'm not quite sure what is causing this behavior.  I've taken a glance at your 
many dependency definitions, and they look correct, so I'm not seeing why 
certain directory paths are required by some repositories.  Required paths like 
this are handled with complex repository dependencies (which you have correctly 
defined), so you may have uncovered a corner-case issue that I didn't encounter 
during development.

This may take a bit of time to figure out and resolve, but I'll let you know as 
soon as we uncover and resolve the problem.

Thanks!

Greg Von Kuster


On Mar 5, 2013, at 1:49 AM, Ira Cooke wrote:

> Hi, 
> 
> I've just uploaded a set of updates to my proteomics tools on the toolshed.  
> These contain my best attempt to implement automatic dependency installation 
> ... and for the most part it seems to work. 
> 
> I have encountered the following issues though
> 
> (1) I don't see any way to ensure that one repository_dependency is installed 
> before all others.  In my case this is important because I have an installer 
> package (galaxy_protk) which needs to be installed before other package 
> installs will work.
> 
> For example .. try installing
> 
> http://toolshed.g2.bx.psu.edu/view/iracooke/omssa
> 
> (2) Running multiple repo installs at once is not reliable (sorry this is so 
> vague).  When I have many dependencies being installed at once I get the 
> following error (for example)
> 
> "Could not cd to 
> ./database/tmp/tmpbIqBW8/galaxy_protk/./database/tmp/tmp3vhKn6/galaxy_protk"
> 
> It looks like a path is not being constructed correctly. 
> 
> If I wait until the currently installed repo is finished .. then try 
> uninstalling and reinstalling it works.
> Sorry this is vague but if you try installing any of the following 
> repositories you should see what I mean
> 
> http://toolshed.g2.bx.psu.edu/view/iracooke/xtandem
> http://toolshed.g2.bx.psu.edu/view/iracooke/msgfplus
> http://toolshed.g2.bx.psu.edu/view/iracooke/tpp_prophets
> 
> Anyway .. and just so it doesn't just sound like I'm complaining ... I think 
> the dependency installation stuff is hugely useful.  It took me a while to 
> figure out all the details .. but I think it makes sense now.
> 
> Cheers
> Ira
> 
> ___
> galaxy-iuc mailing list
> galaxy-...@lists.bx.psu.edu
> http://lists.bx.psu.edu/listinfo/galaxy-iuc

___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:

  http://lists.bx.psu.edu/

[galaxy-dev] Galaxy: Dependency error htseq_count

[Joachim Jacob | VIB |]

Hi Lance,

[apologies for directly emailing to you and not through the galaxy-dev 
list. I will post it on the dev list later on]


I am trying to install htseq-count, but it fails during the dependency 
installation of htseq.


The error I get is:

error: can't create or remove files in install directory

The following error occurred while trying to add or remove files in the
installation directory:

[Errno 2] No such file or directory: 
'/apps/htseq/0.5.3p9/lparsons/htseq_count/f320093f1e8e/lib64/python/test-easy-install-30482.pth'


The installation directory you specified (via --install-dir, --prefix, or
the distutils default setting) was:

/apps/htseq/0.5.3p9/lparsons/htseq_count/f320093f1e8e/lib64/python/

This directory does not currently exist. Please create it and try again, or
choose a different installation directory (using the -d or --install-dir
option).

Any idea whether it is related to my setup (and what I can do about it)? 
Note: only the htseq_count dependency is not being installed.
Thanks for providing the htseq_count wrapper! Now getting it through the 
toolshed will make it again a little closer to a perfect world :-)



Cheers,
Joachim

Joachim Jacob

Rijvisschestraat 120, 9052 Zwijnaarde
Tel: +32 9 244.66.34
Bioinformatics Training and Services (BITS)
http://www.bits.vib.be
@bitsatvib

On 12/17/2012 05:28 PM, Lance Parsons wrote:
Thanks for looking into this Greg.  The environment where I'm having 
this problem is our testing and production systems.


Both environments are CentOS release 5.8 (Final) with Mercurial 
version 2.4.1.


Interestingly, when I tried the same thing with a development setup on 
my OSX box (10.6.8) with Mercurial 2.4 I did not have the same issue. 
One difference is that on my testing/production boxes I do have an 
older version of the tool installed, so perhaps that is the issue.


I also tried uninstalling, but that led to the (expected) error that 
that a previous changeset is installed:
The tool shed repository htseq_count with owner lparsons and changeset 
revision f320093f1e8e was previously installed using changeset 
revision 5d969cb56112. The repository has been uninstalled, however, 
so reinstall the original repository instead of installing it again. 
You can get the latest updates for the repository using the Get 
updates option from the repository's Repository Actions pop-up menu. 
Click here to manage the repository.


I then attempted to reinstall and I got a javascript message the 
cloning failed, and the repo is now "stuck" in the cloning state.  I 
did not see anything in the paster logs regarding this at all.



Here are the steps I took on my local system:

  1. Upload htseq-count v0.3 to local toolshed

  2. Install on local galaxy

  3. Upload htseq-count v0.3-release2 to toolshed

[Errno 2] No such file or directory: 
'/Users/lparsons/Documents/projects/sequencing/galaxy/galaxy-test1/database/community_files/000/repo_3/./tool_data_table_conf.xml.sample'


STDOUT from paster
 htseq-count.xml does not exist!
sam_fa_indices.loc.sample does not exist!
static/images/count_modes.png does not exist!
test-data/htseq-test-paired.bam does not exist!
test-data/htseq-test-paired_counts.tsv does not exist!
test-data/htseq-test-paired_othercounts.tsv does not exist!
test-data/htseq-test.bam does not exist!
test-data/htseq-test.gff does not exist!
test-data/htseq-test.sam does not exist!
test-data/htseq-test_counts.tsv does not exist!
test-data/htseq-test_othercounts.tsv does not exist!
tool_data_table_conf.xml.sample does not exist!
  4. Second attempt to upload
The file 'htseq-count_0.3.release2.tar.gz' has been successfully 
uncompressed and uploaded to the repository. 1 undesirable items (.hg 
.svn .git directories, .DS_Store, hgrc files, etc) were removed from 
the archive. 13 files were removed from the repository root.


  5. Get repository updates
The installed repository named 'htseq_count' has been updated to 
change set revision '74a6a8221337'.




Greg Von Kuster wrote:

Hi Lance,

I tried several scenarios today in an attempt to reproduce this but I 
was not able to.  Have you tried uninstalling the repository (check 
the checkbox to completely uninstall it) and then reinstall it?  I'm 
trying to reproduce this on a mac running mercurial version 2.2.3 and 
Python 2.5.  What's your environment?


Greg Von Kuster


On Dec 13, 2012, at 2:30 PM, Lance Parsons wrote:

Since I've had issues with workflow and the toolshed in the past, I 
have changed my workflow so that my only interaction with the 
toolshed is either via uploading a tarball via the web interface or 
installing a tool into my local Galaxy via the web interface (or 
occasionally just downloading tarballs).


In this specific case, I uploaded a tar.gz file of the entire tool 
wrapper directory.  I did NOT increment the tool version.  In the 
past, incrementing the tool version has generally worked fine.  
However, when I leave the tool version the same, but change 
som

Re: [galaxy-dev] Contributing to genome indexes on rsync server

[Roman Valls]

Hello Brad, Jennifer,

I'm also interested in this initiative since I'm using Brad's code as
a part of a testsuite for a bloom filter:

https://github.com/SciLifeLab/facs/blob/master/facs/utils/galaxy.py

As both of you pointed out, there's a need for some cleanup, there are
some stray files here and there:

-rw-r--r-- 1 vagrant vagrant  34M May 28  2009 .2bit < ???
-rw-r--r-- 1 vagrant vagrant 779M Aug 13  2010 hg19.2bit

In general I think that some good naming conventions in such a good
community resource would be *very* useful.

I will be glad to help! :)

@romanvg on Trello.

Cheers!
Roman

On Sat, Feb 23, 2013 at 8:39 PM, Brad Chapman  wrote:
>
> Jen;
> That sounds great, thanks for your enthusiasm and help organizing this.
> I'm @bradchapman on Trello so feel free to add me to the ticket and let
> me know how I can help. I'm happy to set this up however you feel best:
> looking forward to having a shared repository for all this formatted
> genome data. Thanks again,
> Brad
>
>
>> Hi Brad,
>>
>> I really like this idea. I'm not going to open a ticket yet but talk
>> with Dan/team about some options. We have an alternate directory
>> structure modeled from last fall I'd like to get in place before we
>> start something like this ( is not yet implemented, but it or something
>> similar would be required to properly add in the hg19 GTAK-sort. will
>> involve a bit of other data shuffling w/ .loc changes to keep external
>> links functional ). There are also some other repo ideas in play.
>>
>> Let me get some internal feedback next week, then I'll start a Trello
>> ticket with some basics from our side that we can use to vet a plan to
>> go forward, assuming the rest of the team likes idea. I think test
>> cases/index validation would probably be part of this somehow. And
>> certainly some simplification we be welcomed in the more cluttered dirs,
>> if that can be managed while keeping enough around for reproducibility
>> needs.
>>
>> Very topical, thanks for bring up and offering to help out! These can be
>> great deal of work to create and makes total sense to share. I'll send
>> and update later next week, hopefully with a Trello link so we can get
>> started.
>>
>> Jen
>> Galaxy
>>
>> On 2/21/13 12:43 PM, Brad Chapman wrote:
>>>
>>> Hi all;
>>> Is there a way for community members to contribute indexes to the rsync
>>> server? This resource is awesome and I'm working on migrating the
>>> CloudBioLinux retrieval scripts to use this instead of the custom S3
>>> buckets we'd set up previously:
>>>
>>> https://github.com/chapmanb/cloudbiolinux/blob/master/cloudbio/biodata/galaxy.py
>>>
>>> It's great to have this as a public shared resource and I'd like to be
>>> able to contribute back. From an initial pass, here are the things I'd
>>> like to do:
>>>
>>> - Include bowtie2 indexes for more genomes.
>>>
>>> - Include novoalign indexes for a number of commonly used genomes.
>>>
>>> - Clean up hg19 to include a full canonically sorted hg19, with indexes.
>>>Broad has a nice version prepped so GATK will be happy with it, and
>>>you need to stick with this ordering if you're ever going to use a
>>>GATK tool on it. Right now there is a partial hg19canon (without the
>>>random/haplotype chromosomes) and the structure is a bit complex.
>>>
>>> What's the best way to contribute these? Right now I have a lot of the
>>> indexes on S3. For instance, the hg19 indexes are here:
>>>
>>> https://s3.amazonaws.com/biodata/genomes/hg19-bowtie.tar.xz
>>> https://s3.amazonaws.com/biodata/genomes/hg19-bowtie2.tar.xz
>>> https://s3.amazonaws.com/biodata/genomes/hg19-bwa.tar.xz
>>> https://s3.amazonaws.com/biodata/genomes/hg19-novoalign.tar.xz
>>> https://s3.amazonaws.com/biodata/genomes/hg19-seq.tar.xz
>>> https://s3.amazonaws.com/biodata/genomes/hg19-ucsc.tar.xz
>>>
>>> I'm happy to format these differently or upload somewhere that would
>>> make it easy to include. Thanks again for setting this up, I'm looking
>>> forward to working off a shared repository of data,
>>> Brad
>>> ___
>>> Please keep all replies on the list by using "reply all"
>>> in your mail client.  To manage your subscriptions to this
>>> and other Galaxy lists, please use the interface at:
>>>
>>>http://lists.bx.psu.edu/
>>>
>>
>> --
>> Jennifer Hillman-Jackson
>> Galaxy Support and Training
>> http://galaxyproject.org
> ___
> Please keep all replies on the list by using "reply all"
> in your mail client.  To manage your subscriptions to this
> and other Galaxy lists, please use the interface at:
>
>   http://lists.bx.psu.edu/
___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:

  http://lists.bx.psu.edu/

Re: [galaxy-dev] [galaxy-iuc] Installing repositories in a specific order

[Peter Cock]

On Tue, Mar 5, 2013 at 10:08 AM, Ira Cooke  wrote:
> Hi Peter,
>
> The problem is that I have multiple repositories in my repository_dependency
> file ... but they need to be installed in a specific order.  I can't seem to 
> get
> galaxy to install them in the correct order.
>
> Ira
>
> PS: Sorry about the totally irrelevant subject line ... I've changed it.

Ah. I think I see now.

So for example repository D depends on A, B and C - but you must
install A before B before C (before D)? In this case it sounds like
you could define A as a dependency of B etc.

Peter
___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:

  http://lists.bx.psu.edu/

Re: [galaxy-dev] [galaxy-iuc] Proteomics category on test toolshed

[Peter Cock]

On Tue, Mar 5, 2013 at 6:49 AM, Ira Cooke  wrote:
> Hi,
>
> I've just uploaded a set of updates to my proteomics tools on the toolshed.
> These contain my best attempt to implement automatic dependency installation
> ... and for the most part it seems to work.
>
> I have encountered the following issues though
>
> (1) I don't see any way to ensure that one repository_dependency is
> installed before all others.  In my case this is important because I have an
> installer package (galaxy_protk) which needs to be installed before other
> package installs will work.
>
> For example .. try installing
>
> http://toolshed.g2.bx.psu.edu/view/iracooke/omssa

I think in this case you just need to use a repository_dependencies.xml
file (but note currently this only lets you point at other repositories in
the same Tool Shed instance, i.e. http://toolshed.g2.bx.psu.edu/ here):
http://wiki.galaxyproject.org/DefiningRepositoryDependencies

Peter
___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:

  http://lists.bx.psu.edu/