[galaxy-dev] Cluster jobs running as real user

[Shane Sturrock]

Has anyone had any success in getting cluster jobs running as the real user?  
I've got our server happily working with LSF via drmaa but all jobs run as the 
galaxy user and I would like to be able to separate out the jobs on a per user 
basis.  I've followed the instructions at 
https://wiki.galaxyproject.org/Admin/Config/Performance/Cluster#Submitting_Jobs_as_the_Real_User
but I'm having no luck with it so was wondering if someone would know how I 
might debug it, like where could I find the script it is trying to execute?  
All I'm getting is code 18: invalid LSF job id although it does seem to be 
creating files in my databases/tmp directory owned by the user.

Shane

--
Dr Shane Sturrock
shane.sturr...@biomatters.com
Senior Scientist
Tel: +64 (0) 9 379 5064
76 Anzac Ave
Auckland
New Zealand

___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  http://lists.bx.psu.edu/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/

[galaxy-dev] read_bits64 error when loading large genomes/data into trackster

[Raj Ayyampalayam]

Hello,

I am trying to visualize a large genome (Large number of scaffolds) and 
a large (bam file) in trackster on our local galaxy instance (running 
release_2014.02.10).

When ever I try to do the above I see the following error in the logs:

  File "bbi_file.pyx", line 215, in bx.bbi.bbi_file.BBIFile.query 
(lib/bx/bbi/bbi_file.c:5596)
  File "bbi_file.pyx", line 222, in bx.bbi.bbi_file.BBIFile.query 
(lib/bx/bbi/bbi_file.c:5210)
  File "bbi_file.pyx", line 183, in bx.bbi.bbi_file.BBIFile.summarize 
(lib/bx/bbi/bbi_file.c:4475)
  File "bbi_file.pyx", line 248, in 
bx.bbi.bbi_file.BBIFile._get_chrom_id_and_size (lib/bx/bbi/bbi_file.c:5656)
  File "bpt_file.pyx", line 76, in bx.bbi.bpt_file.BPTFile.find 
(lib/bx/bbi/bpt_file.c:1388)
  File "bpt_file.pyx", line 55, in bx.bbi.bpt_file.BPTFile.r_find 
(lib/bx/bbi/bpt_file.c:1154)

AttributeError: 'BinaryFileReader' object has no attribute 'read_bits64'

Trackster works OK when I load smaller data sets.

It seems that there was a fix for this in bx-python code, as per mail 
from Jeremy 
(http://dev.list.galaxyproject.org/trackster-error-for-viewing-rat-data-rn5-tp4662664p4662672.html).

How do I get the fixed code into my galaxy instance?

Thanks,
-Raj


___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  http://lists.bx.psu.edu/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/

Re: [galaxy-dev] How to install bowtie2 tool in galaxy

[Ravi Alla]

Bjoern,
This is getting frustrating.
There are three places bowtie2_indices.loc file is expected. I really don't 
know which one is the one I should modify.

galaxy-dist/tool-data/bowtie2_indices.loc
galaxy-dist/tool-data/toolshed.g2.bx.psu.edu/repos/devteam/bowtie2/96d2e31a3938/bowtie2_indices.loc
../shed_tools/toolshed.g2.bx.psu.edu/repos/devteam/bowtie2/96d2e31a3938/bowtie2/tool-data/bowtie2_indices.loc

What is the difference between these 3 files?

I changed the 2nd file to include the path to the indices and this shows up as 
a valid preloaded index for tophat2 but does not for bowtie2. I also changed 
the normal bowtie_indices.loc file at the second location and this works and I 
can see the bowtie1 indices for that bowtie tool. Currently only the bowtie2 
indices are acting up.

I wish this was easier.
Thanks
Ravi.
On Mar 4, 2014, at 11:28 AM, Björn Grüning  wrote:

> Hi,
> 
> can you check if the file tool_data_table_conf.xml contains an entry about 
> bowtie2_indexes ...  or the table shed_data_table_conf.xml ... in one of them 
> should be one entry with bowtie2_indexes.
> 
> That warning should not be there I think:
> 
> > galaxy.tools.parameters.dynamic_options WARNING 2014-03-04 10:14:33,335 
> > Data table named 'bowtie2_indexes' is required by tool but not configured
> 
> Hope you can figure it out,
> Bjoern
> 
> Am 04.03.2014 19:19, schrieb Ravi Alla:
>> Hi Bjoern,
>> Please find the bowtie2 and bowtie .loc files. In the paster.log file I see 
>> these entries for bowtie
>> 
>> galaxy.tools.data DEBUG 2014-03-04 10:14:11,048 Loaded tool data table 
>> 'bowtie_indexes'
>> galaxy.tools.data DEBUG 2014-03-04 10:14:11,051 Loading another instance of 
>> data table 'bowtie_indexes', attempting to merge content.
>> galaxy.tools.parameters.dynamic_options WARNING 2014-03-04 10:14:33,335 Data 
>> table named 'bowtie2_indexes' is required by tool but not configured
>> 
>> And ls -l /global/referenceData/databases/bowtie2/hg19 is
>> drwxr-xr-x 2 ralla cgrl  2048 Sep 10 11:14 .
>> drwxr-xr-x 3 ralla cgrl  2048 Sep 10 11:06 ..
>> -rw-rw-r-- 1 ralla cgrl 960018873 May  2  2012 hg19.1.bt2
>> -rw-rw-r-- 1 ralla cgrl 716863572 May  2  2012 hg19.2.bt2
>> -rw-rw-r-- 1 ralla cgrl  3833 May  2  2012 hg19.3.bt2
>> -rw-rw-r-- 1 ralla cgrl 716863565 May  2  2012 hg19.4.bt2
>> -rw-rw-r-- 1 ralla cgrl 960018873 May  2  2012 hg19.rev.1.bt2
>> -rw-rw-r-- 1 ralla cgrl 716863572 May  2  2012 hg19.rev.2.bt2
>> 
>> and ls -l /global/referenceData/databases/bowtie/hg19 is
>> drwxr-xr-x 2 ralla cgrl  2048 Mar  4 10:07 .
>> drwxr-xr-x 3 ralla cgrl  2048 Nov 30  2012 ..
>> -rw-rw-r-- 1 ralla cgrl 821725563 Nov 13  2009 hg19.1.ebwt
>> -rw-rw-r-- 1 ralla cgrl 357667968 Nov 13  2009 hg19.2.ebwt
>> -rw-rw-r-- 1 ralla cgrl  3284 Nov 13  2009 hg19.3.ebwt
>> -rw-rw-r-- 1 ralla cgrl 715335926 Nov 13  2009 hg19.4.ebwt
>> lrwxrwxrwx 1 ralla cgrl45 Aug 21  2013 hg19.fa -> 
>> /global/referenceData/genomes/hs/hg19/hg19.fa
>> -rw-rw-r-- 1 ralla cgrl 821725563 Nov 13  2009 hg19.rev.1.ebwt
>> -rw-rw-r-- 1 ralla cgrl 357667968 Nov 13  2009 hg19.rev.2.ebwt
>> 
>> Thanks for looking into this.
>> Ravi.
>> On Mar 4, 2014, at 9:13 AM, Björn Grüning  wrote:
>> 
>>> Hi Ravi,
>>> 
>>> can you attach the loc file, do you see anything in the Galaxy log files 
>>> about bowtie2, try grepping for "bowtie2".
>>> 
>>> Cheers,
>>> Bjoern
>>> 
>>> Am 04.03.2014 18:07, schrieb Ravi Alla:
 Hi Bjoern,
 Thank you for your clarifications. I have tried installing bowtie2 using 
 both variations, once with both wrapper and dependencies and once with 
 only wrapper (since I have dependencies installed on the system). In 
 either case even after editing the .loc file I cannot see the indices in 
 galaxy. I tried the same edits to .loc file with bwa and the indices show 
 right up, but not with bowtie2 and even bowtie for that matter. I really 
 don't know what to do about this.
 Thanks
 Ravi
 On Mar 4, 2014, at 3:06 AM, Björn Grüning  
 wrote:
 
> Hi Ravi,
> 
>> Hi guys,
>> I am new to galaxy and am in the process of setting it up on a cluster 
>> with some help. I am trying to install bowtie2 to my local galaxy 
>> through the toolshed. I have a few questions.
>> 
>> - Which bowtie2 should I install? When I search for bowtie2 a bunch of 
>> results come up?
> 
> Only two, or? Please make sure you are using the main toolshed. bowtie2 
> and package_bowtie2_2_1_0 are connected together. One is the binary and 
> the other contains the wrapper. Use devteam respositories is always a 
> good choice.
> 
>> - What are repository dependencies? Why would I need these if bowtie2 is 
>> already installed on my system?
> 
> repository dependecies containing all dependencies of your wrappers, that 
> can be some binaries, but also R, Perl, python libraries that are needed 
> to execute your wrappers. We recomme

[galaxy-dev] BOSC 2014 Call for Abstracts

[Dave Clements]

Hello all

The call for abstracts for BOSC 2014 is out.  See below for details.

If you have questions I'm sure that Peter, Brad, Chris, and Hans-Rudolf,
all frequent contributors to this list, would be answer them.  And glad to
see that GigaScience, a GCC sponsor, is also sponsoring BOSC this year.

Cheers,

Dave C.


Call for Abstracts for the 15th Annual Bioinformatics Open Source
Conference (BOSC 2014)
A Special Interest Group (SIG) of ISMB 2014

Dates: July 11-12, 2014
Location: Boston, MA, USA
Web site: http://www.open-bio.org/wiki/BOSC_2014
Email: b...@open-bio.org
BOSC announcements mailing list:
http://lists.open-bio.org/mailman/listinfo/bosc-announce

Important Dates:
March 24, 2014: Registration opens for ISMB and BOSC
(https://www.iscb.org/ismb2014-registration)
April 4, 2014: Deadline for submitting BOSC abstracts
(http://www.open-bio.org/wiki/BOSC_Abstract_Submission)
May 1, 204: Notification of accepted talk abstracts emailed to authors
July 9-10, 2014: Codefest 2014, Boston
(http://www.open-bio.org/wiki/Codefest_2014)
July 11-12, 2014: BOSC 2014, Boston (http://www.open-bio.org/wiki/BOSC_2014)
July 11-15, 2014: ISMB 2014, Boston

The Bioinformatics Open Source Conference (BOSC) covers the wide range
of open source bioinformatics software being developed, and
encompasses the growing movement of Open Science, with its focus on
transparency, reproducibility, and data provenance.  We welcome
submissions relating to all aspects of bioinformatics and open science
software, including new computational methods, reusable software
components, visualization, interoperability, and other approaches that
help to advance research in the biomolecular sciences. Two full days
of talks, posters, panel discussions, and informal discussion groups
will enable BOSC attendees to interact with other developers and share
ideas and code, as well as learning about some of the latest
developments in the field of open source bioinformatics. BOSC is
sponsored by the Open Bioinformatics Foundation, a non-profit,
volunteer-run group dedicated to promoting the practice and philosophy
of Open Source software development and Open Scien!
 ce within the biological research community.

We invite you to submit one-page abstracts for talks and posters.
This year's session topics are:
   Open Science and Reproducible Research
   Software Interoperability
   Genome-scale Data and Beyond
   Visualization
   Translational Bioinformatics
   Bioinformatics Open Source Libraries and Projects

Once again we thank Eagle Genomics for sponsoring the BOSC Student
Travel Awards, and welcome the open access journal GigaScience as a
new sponsor for BOSC 2014.

BOSC 2014 Organizing Committee:
Nomi Harris and Peter Cock (co-chairs), Raoul Jean Pierre Bonnal, Brad
Chapman, Robert Davey, Christopher Fields, Hans-Rudolf Hotz, Hilmar
Lapp


___
Members mailing list
memb...@lists.open-bio.org
http://lists.open-bio.org/mailman/listinfo/members
___
Biopython mailing list  -  biopyt...@lists.open-bio.org
http://lists.open-bio.org/mailman/listinfo/biopython



-- 
http://galaxyproject.org/
http://getgalaxy.org/
http://usegalaxy.org/
http://wiki.galaxyproject.org/
___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  http://lists.bx.psu.edu/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/

Re: [galaxy-dev] pbs runner deserializes server names as unicode

[Eric Badger]

Hey John,

We're using job_conf.xml, which does not specify an encoding. It doesn't
seem to be the XML parsing at issue; if you omit the restarting of
galaxy step, jobs go through just fine. Only when galaxy is restarted
while the job is running (and destination_params must therefore be
reloaded) does the issue appear. 

The provided hack does the trick; I'd done up something similar myself
but felt like I might've been going about it the wrong way. I'll forge
ahead with it though if it looks to be the right way. 

I'm getting this, by the way, on a freshly cloned galaxy too, not just
on the existing MSI instances. 

Thanks,
Eric

On Tue, Mar 04, 2014 at 01:30:00PM -0600, John Chilton wrote:
> Hey Eric,
> 
> Odd - is MSI still using tool runner URLs in universe_wsgi.ini or have
> you guys transitioned to job_conf.xml. If you are using a job_conf.xml
> file - out of curiosity are you explicitly declaring an XML encoding
> at the top of it? Not that this would be the wrong thing to do - I am
> just trying to understand why you guys are the first to encounter this
> bug.
> 
> Regardless, does this hack get you anywhere?
> 
> https://gist.github.com/jmchilton/9353654
> 
> -John
> 
> 
> On Tue, Mar 4, 2014 at 12:46 PM, Eric Badger  wrote:
> > Hi there,
> >
> > I've run into an issue with the pbs runner involving pbs server name
> > strings becoming 'unicode' rather than 'str'. The pbs_python lib can't
> > pass unicode strings down to the C library, which results in this
> > failure:
> >
> > https://gist.github.com/anonymous/9352218
> >
> > The very top line of log output in the above gist is due to a line I added 
> > to
> > runners/pbs.py:
> >
> > https://gist.github.com/anonymous/9352101#file-pbs-py-L16
> >
> > I can reproduce this as follows:
> >
> > 1. Start a job using pbs runner.
> > 2. While the job is still running, restart galaxy.
> > 3. Error now appears in the logs.
> >
> > It seems that this is happening because, when deserializing
> > job.destination_params from the database, the strings are converted to
> > unicode. The jobs in this state are effectively stuck, since galaxy
> > can't connect to check on their status.
> >
> > I'm not sure where the best place to address this would be; thoughts?
> >
> > This is on an up to date galaxy-dist with Torque 2.5.13.
> >
> > Thanks,
> > Eric
> >
> > ___
> > Please keep all replies on the list by using "reply all"
> > in your mail client.  To manage your subscriptions to this
> > and other Galaxy lists, please use the interface at:
> >   http://lists.bx.psu.edu/
> >
> > To search Galaxy mailing lists use the unified search at:
> >   http://galaxyproject.org/search/mailinglists/
___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  http://lists.bx.psu.edu/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/

Re: [galaxy-dev] pbs runner deserializes server names as unicode

[John Chilton]

Hey Eric,

Odd - is MSI still using tool runner URLs in universe_wsgi.ini or have
you guys transitioned to job_conf.xml. If you are using a job_conf.xml
file - out of curiosity are you explicitly declaring an XML encoding
at the top of it? Not that this would be the wrong thing to do - I am
just trying to understand why you guys are the first to encounter this
bug.

Regardless, does this hack get you anywhere?

https://gist.github.com/jmchilton/9353654

-John


On Tue, Mar 4, 2014 at 12:46 PM, Eric Badger  wrote:
> Hi there,
>
> I've run into an issue with the pbs runner involving pbs server name
> strings becoming 'unicode' rather than 'str'. The pbs_python lib can't
> pass unicode strings down to the C library, which results in this
> failure:
>
> https://gist.github.com/anonymous/9352218
>
> The very top line of log output in the above gist is due to a line I added to
> runners/pbs.py:
>
> https://gist.github.com/anonymous/9352101#file-pbs-py-L16
>
> I can reproduce this as follows:
>
> 1. Start a job using pbs runner.
> 2. While the job is still running, restart galaxy.
> 3. Error now appears in the logs.
>
> It seems that this is happening because, when deserializing
> job.destination_params from the database, the strings are converted to
> unicode. The jobs in this state are effectively stuck, since galaxy
> can't connect to check on their status.
>
> I'm not sure where the best place to address this would be; thoughts?
>
> This is on an up to date galaxy-dist with Torque 2.5.13.
>
> Thanks,
> Eric
>
> ___
> Please keep all replies on the list by using "reply all"
> in your mail client.  To manage your subscriptions to this
> and other Galaxy lists, please use the interface at:
>   http://lists.bx.psu.edu/
>
> To search Galaxy mailing lists use the unified search at:
>   http://galaxyproject.org/search/mailinglists/
___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  http://lists.bx.psu.edu/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/

Re: [galaxy-dev] How to install bowtie2 tool in galaxy

[Björn Grüning]

Hi,

can you check if the file tool_data_table_conf.xml contains an entry 
about bowtie2_indexes ...  or the table shed_data_table_conf.xml ... in 
one of them should be one entry with bowtie2_indexes.


That warning should not be there I think:

> galaxy.tools.parameters.dynamic_options WARNING 2014-03-04 
10:14:33,335 Data table named 'bowtie2_indexes' is required by tool but 
not configured


Hope you can figure it out,
Bjoern

Am 04.03.2014 19:19, schrieb Ravi Alla:

Hi Bjoern,
Please find the bowtie2 and bowtie .loc files. In the paster.log file I see 
these entries for bowtie

galaxy.tools.data DEBUG 2014-03-04 10:14:11,048 Loaded tool data table 
'bowtie_indexes'
galaxy.tools.data DEBUG 2014-03-04 10:14:11,051 Loading another instance of 
data table 'bowtie_indexes', attempting to merge content.
galaxy.tools.parameters.dynamic_options WARNING 2014-03-04 10:14:33,335 Data 
table named 'bowtie2_indexes' is required by tool but not configured

And ls -l /global/referenceData/databases/bowtie2/hg19 is
drwxr-xr-x 2 ralla cgrl  2048 Sep 10 11:14 .
drwxr-xr-x 3 ralla cgrl  2048 Sep 10 11:06 ..
-rw-rw-r-- 1 ralla cgrl 960018873 May  2  2012 hg19.1.bt2
-rw-rw-r-- 1 ralla cgrl 716863572 May  2  2012 hg19.2.bt2
-rw-rw-r-- 1 ralla cgrl  3833 May  2  2012 hg19.3.bt2
-rw-rw-r-- 1 ralla cgrl 716863565 May  2  2012 hg19.4.bt2
-rw-rw-r-- 1 ralla cgrl 960018873 May  2  2012 hg19.rev.1.bt2
-rw-rw-r-- 1 ralla cgrl 716863572 May  2  2012 hg19.rev.2.bt2

and ls -l /global/referenceData/databases/bowtie/hg19 is
drwxr-xr-x 2 ralla cgrl  2048 Mar  4 10:07 .
drwxr-xr-x 3 ralla cgrl  2048 Nov 30  2012 ..
-rw-rw-r-- 1 ralla cgrl 821725563 Nov 13  2009 hg19.1.ebwt
-rw-rw-r-- 1 ralla cgrl 357667968 Nov 13  2009 hg19.2.ebwt
-rw-rw-r-- 1 ralla cgrl  3284 Nov 13  2009 hg19.3.ebwt
-rw-rw-r-- 1 ralla cgrl 715335926 Nov 13  2009 hg19.4.ebwt
lrwxrwxrwx 1 ralla cgrl45 Aug 21  2013 hg19.fa -> 
/global/referenceData/genomes/hs/hg19/hg19.fa
-rw-rw-r-- 1 ralla cgrl 821725563 Nov 13  2009 hg19.rev.1.ebwt
-rw-rw-r-- 1 ralla cgrl 357667968 Nov 13  2009 hg19.rev.2.ebwt

Thanks for looking into this.
Ravi.
On Mar 4, 2014, at 9:13 AM, Björn Grüning  wrote:


Hi Ravi,

can you attach the loc file, do you see anything in the Galaxy log files about bowtie2, 
try grepping for "bowtie2".

Cheers,
Bjoern

Am 04.03.2014 18:07, schrieb Ravi Alla:

Hi Bjoern,
Thank you for your clarifications. I have tried installing bowtie2 using both 
variations, once with both wrapper and dependencies and once with only wrapper 
(since I have dependencies installed on the system). In either case even after 
editing the .loc file I cannot see the indices in galaxy. I tried the same 
edits to .loc file with bwa and the indices show right up, but not with bowtie2 
and even bowtie for that matter. I really don't know what to do about this.
Thanks
Ravi
On Mar 4, 2014, at 3:06 AM, Björn Grüning  wrote:


Hi Ravi,


Hi guys,
I am new to galaxy and am in the process of setting it up on a cluster with 
some help. I am trying to install bowtie2 to my local galaxy through the 
toolshed. I have a few questions.

- Which bowtie2 should I install? When I search for bowtie2 a bunch of results 
come up?


Only two, or? Please make sure you are using the main toolshed. bowtie2 and 
package_bowtie2_2_1_0 are connected together. One is the binary and the other 
contains the wrapper. Use devteam respositories is always a good choice.


- What are repository dependencies? Why would I need these if bowtie2 is 
already installed on my system?


repository dependecies containing all dependencies of your wrappers, that can 
be some binaries, but also R, Perl, python libraries that are needed to execute 
your wrappers. We recommend you to use the dependencies because then you have 
full control over all your used versions (wrapper, dependencies, tool) and that 
enables reproducibility. To put in other words: With the toolshed you can have 
different tool and wrappers versions with different dependencies at the same 
time.


- I have been installing bowtie2 by unchecking the repository dependencies and 
tool dependencies boxes. Is this correct?


It will work, if you have a system installed version. But you need to take care 
about binary updates by your own. And reproducibility of your results is not 
guaranteed.


- Finally when I install bowtie2 this way it creates 2 bowtie2_indices.loc 
files, one in galaxy/tool-data and the other in 
shed_tools/toolshed.g2.gx.psu.edu/dev/bowtie2, which one do I need to edit to 
point to my index files? No matter which one I change I can't seem to see the 
indices.


That is strange. Are you sure you don't have a spelling mistake in it. It 
should be the bowtie2_indices.los I think. I will CC Greg, he is one of the 
Tool Shed Developers and should know more about it.
Please do not forget to restart your Galaxy instance once you have update the 
*loc files.


I hope someone on here can help me out.


Hope it helped a little bi

[galaxy-dev] pbs runner deserializes server names as unicode

[Eric Badger]

Hi there,

I've run into an issue with the pbs runner involving pbs server name
strings becoming 'unicode' rather than 'str'. The pbs_python lib can't
pass unicode strings down to the C library, which results in this
failure:

https://gist.github.com/anonymous/9352218

The very top line of log output in the above gist is due to a line I added to
runners/pbs.py:

https://gist.github.com/anonymous/9352101#file-pbs-py-L16

I can reproduce this as follows:

1. Start a job using pbs runner.
2. While the job is still running, restart galaxy.
3. Error now appears in the logs.

It seems that this is happening because, when deserializing
job.destination_params from the database, the strings are converted to
unicode. The jobs in this state are effectively stuck, since galaxy
can't connect to check on their status.

I'm not sure where the best place to address this would be; thoughts?

This is on an up to date galaxy-dist with Torque 2.5.13.

Thanks,
Eric

___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  http://lists.bx.psu.edu/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/

Re: [galaxy-dev] How to install bowtie2 tool in galaxy

[Ravi Alla]

Hi Bjoern,
Please find the bowtie2 and bowtie .loc files. In the paster.log file I see 
these entries for bowtie

galaxy.tools.data DEBUG 2014-03-04 10:14:11,048 Loaded tool data table 
'bowtie_indexes'
galaxy.tools.data DEBUG 2014-03-04 10:14:11,051 Loading another instance of 
data table 'bowtie_indexes', attempting to merge content.
galaxy.tools.parameters.dynamic_options WARNING 2014-03-04 10:14:33,335 Data 
table named 'bowtie2_indexes' is required by tool but not configured

And ls -l /global/referenceData/databases/bowtie2/hg19 is 
drwxr-xr-x 2 ralla cgrl  2048 Sep 10 11:14 .
drwxr-xr-x 3 ralla cgrl  2048 Sep 10 11:06 ..
-rw-rw-r-- 1 ralla cgrl 960018873 May  2  2012 hg19.1.bt2
-rw-rw-r-- 1 ralla cgrl 716863572 May  2  2012 hg19.2.bt2
-rw-rw-r-- 1 ralla cgrl  3833 May  2  2012 hg19.3.bt2
-rw-rw-r-- 1 ralla cgrl 716863565 May  2  2012 hg19.4.bt2
-rw-rw-r-- 1 ralla cgrl 960018873 May  2  2012 hg19.rev.1.bt2
-rw-rw-r-- 1 ralla cgrl 716863572 May  2  2012 hg19.rev.2.bt2

and ls -l /global/referenceData/databases/bowtie/hg19 is
drwxr-xr-x 2 ralla cgrl  2048 Mar  4 10:07 .
drwxr-xr-x 3 ralla cgrl  2048 Nov 30  2012 ..
-rw-rw-r-- 1 ralla cgrl 821725563 Nov 13  2009 hg19.1.ebwt
-rw-rw-r-- 1 ralla cgrl 357667968 Nov 13  2009 hg19.2.ebwt
-rw-rw-r-- 1 ralla cgrl  3284 Nov 13  2009 hg19.3.ebwt
-rw-rw-r-- 1 ralla cgrl 715335926 Nov 13  2009 hg19.4.ebwt
lrwxrwxrwx 1 ralla cgrl45 Aug 21  2013 hg19.fa -> 
/global/referenceData/genomes/hs/hg19/hg19.fa
-rw-rw-r-- 1 ralla cgrl 821725563 Nov 13  2009 hg19.rev.1.ebwt
-rw-rw-r-- 1 ralla cgrl 357667968 Nov 13  2009 hg19.rev.2.ebwt

Thanks for looking into this.
Ravi.
On Mar 4, 2014, at 9:13 AM, Björn Grüning  wrote:

> Hi Ravi,
> 
> can you attach the loc file, do you see anything in the Galaxy log files 
> about bowtie2, try grepping for "bowtie2".
> 
> Cheers,
> Bjoern
> 
> Am 04.03.2014 18:07, schrieb Ravi Alla:
>> Hi Bjoern,
>> Thank you for your clarifications. I have tried installing bowtie2 using 
>> both variations, once with both wrapper and dependencies and once with only 
>> wrapper (since I have dependencies installed on the system). In either case 
>> even after editing the .loc file I cannot see the indices in galaxy. I tried 
>> the same edits to .loc file with bwa and the indices show right up, but not 
>> with bowtie2 and even bowtie for that matter. I really don't know what to do 
>> about this.
>> Thanks
>> Ravi
>> On Mar 4, 2014, at 3:06 AM, Björn Grüning  wrote:
>> 
>>> Hi Ravi,
>>> 
 Hi guys,
 I am new to galaxy and am in the process of setting it up on a cluster 
 with some help. I am trying to install bowtie2 to my local galaxy through 
 the toolshed. I have a few questions.
 
 - Which bowtie2 should I install? When I search for bowtie2 a bunch of 
 results come up?
>>> 
>>> Only two, or? Please make sure you are using the main toolshed. bowtie2 and 
>>> package_bowtie2_2_1_0 are connected together. One is the binary and the 
>>> other contains the wrapper. Use devteam respositories is always a good 
>>> choice.
>>> 
 - What are repository dependencies? Why would I need these if bowtie2 is 
 already installed on my system?
>>> 
>>> repository dependecies containing all dependencies of your wrappers, that 
>>> can be some binaries, but also R, Perl, python libraries that are needed to 
>>> execute your wrappers. We recommend you to use the dependencies because 
>>> then you have full control over all your used versions (wrapper, 
>>> dependencies, tool) and that enables reproducibility. To put in other 
>>> words: With the toolshed you can have different tool and wrappers versions 
>>> with different dependencies at the same time.
>>> 
 - I have been installing bowtie2 by unchecking the repository dependencies 
 and tool dependencies boxes. Is this correct?
>>> 
>>> It will work, if you have a system installed version. But you need to take 
>>> care about binary updates by your own. And reproducibility of your results 
>>> is not guaranteed.
>>> 
 - Finally when I install bowtie2 this way it creates 2 bowtie2_indices.loc 
 files, one in galaxy/tool-data and the other in 
 shed_tools/toolshed.g2.gx.psu.edu/dev/bowtie2, which one do I need to edit 
 to point to my index files? No matter which one I change I can't seem to 
 see the indices.
>>> 
>>> That is strange. Are you sure you don't have a spelling mistake in it. It 
>>> should be the bowtie2_indices.los I think. I will CC Greg, he is one of the 
>>> Tool Shed Developers and should know more about it.
>>> Please do not forget to restart your Galaxy instance once you have update 
>>> the *loc files.
>>> 
 I hope someone on here can help me out.
>>> 
>>> Hope it helped a little bit,
>>> Bjoern
>>> 
 Thanks a lot
 Ravi.
 ___
 Please keep all replies on the list by using "reply all"
 in your mail client.  To manage your subscri

Re: [galaxy-dev] How to install bowtie2 tool in galaxy

[Björn Grüning]

Hi Ravi,

can you attach the loc file, do you see anything in the Galaxy log files 
about bowtie2, try grepping for "bowtie2".


Cheers,
Bjoern

Am 04.03.2014 18:07, schrieb Ravi Alla:

Hi Bjoern,
Thank you for your clarifications. I have tried installing bowtie2 using both 
variations, once with both wrapper and dependencies and once with only wrapper 
(since I have dependencies installed on the system). In either case even after 
editing the .loc file I cannot see the indices in galaxy. I tried the same 
edits to .loc file with bwa and the indices show right up, but not with bowtie2 
and even bowtie for that matter. I really don't know what to do about this.
Thanks
Ravi
On Mar 4, 2014, at 3:06 AM, Björn Grüning  wrote:


Hi Ravi,


Hi guys,
I am new to galaxy and am in the process of setting it up on a cluster with 
some help. I am trying to install bowtie2 to my local galaxy through the 
toolshed. I have a few questions.

- Which bowtie2 should I install? When I search for bowtie2 a bunch of results 
come up?


Only two, or? Please make sure you are using the main toolshed. bowtie2 and 
package_bowtie2_2_1_0 are connected together. One is the binary and the other 
contains the wrapper. Use devteam respositories is always a good choice.


- What are repository dependencies? Why would I need these if bowtie2 is 
already installed on my system?


repository dependecies containing all dependencies of your wrappers, that can 
be some binaries, but also R, Perl, python libraries that are needed to execute 
your wrappers. We recommend you to use the dependencies because then you have 
full control over all your used versions (wrapper, dependencies, tool) and that 
enables reproducibility. To put in other words: With the toolshed you can have 
different tool and wrappers versions with different dependencies at the same 
time.


- I have been installing bowtie2 by unchecking the repository dependencies and 
tool dependencies boxes. Is this correct?


It will work, if you have a system installed version. But you need to take care 
about binary updates by your own. And reproducibility of your results is not 
guaranteed.


- Finally when I install bowtie2 this way it creates 2 bowtie2_indices.loc 
files, one in galaxy/tool-data and the other in 
shed_tools/toolshed.g2.gx.psu.edu/dev/bowtie2, which one do I need to edit to 
point to my index files? No matter which one I change I can't seem to see the 
indices.


That is strange. Are you sure you don't have a spelling mistake in it. It 
should be the bowtie2_indices.los I think. I will CC Greg, he is one of the 
Tool Shed Developers and should know more about it.
Please do not forget to restart your Galaxy instance once you have update the 
*loc files.


I hope someone on here can help me out.


Hope it helped a little bit,
Bjoern


Thanks a lot
Ravi.
___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
   http://lists.bx.psu.edu/

To search Galaxy mailing lists use the unified search at:
   http://galaxyproject.org/search/mailinglists/




___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
 http://lists.bx.psu.edu/

To search Galaxy mailing lists use the unified search at:
 http://galaxyproject.org/search/mailinglists/

Re: [galaxy-dev] How to install bowtie2 tool in galaxy

[Ravi Alla]

Hi Bjoern,
Thank you for your clarifications. I have tried installing bowtie2 using both 
variations, once with both wrapper and dependencies and once with only wrapper 
(since I have dependencies installed on the system). In either case even after 
editing the .loc file I cannot see the indices in galaxy. I tried the same 
edits to .loc file with bwa and the indices show right up, but not with bowtie2 
and even bowtie for that matter. I really don't know what to do about this.
Thanks
Ravi
On Mar 4, 2014, at 3:06 AM, Björn Grüning  wrote:

> Hi Ravi,
> 
>> Hi guys,
>> I am new to galaxy and am in the process of setting it up on a cluster with 
>> some help. I am trying to install bowtie2 to my local galaxy through the 
>> toolshed. I have a few questions.
>> 
>> - Which bowtie2 should I install? When I search for bowtie2 a bunch of 
>> results come up?
> 
> Only two, or? Please make sure you are using the main toolshed. bowtie2 and 
> package_bowtie2_2_1_0 are connected together. One is the binary and the other 
> contains the wrapper. Use devteam respositories is always a good choice.
> 
>> - What are repository dependencies? Why would I need these if bowtie2 is 
>> already installed on my system?
> 
> repository dependecies containing all dependencies of your wrappers, that can 
> be some binaries, but also R, Perl, python libraries that are needed to 
> execute your wrappers. We recommend you to use the dependencies because then 
> you have full control over all your used versions (wrapper, dependencies, 
> tool) and that enables reproducibility. To put in other words: With the 
> toolshed you can have different tool and wrappers versions with different 
> dependencies at the same time.
> 
>> - I have been installing bowtie2 by unchecking the repository dependencies 
>> and tool dependencies boxes. Is this correct?
> 
> It will work, if you have a system installed version. But you need to take 
> care about binary updates by your own. And reproducibility of your results is 
> not guaranteed.
> 
>> - Finally when I install bowtie2 this way it creates 2 bowtie2_indices.loc 
>> files, one in galaxy/tool-data and the other in 
>> shed_tools/toolshed.g2.gx.psu.edu/dev/bowtie2, which one do I need to edit 
>> to point to my index files? No matter which one I change I can't seem to see 
>> the indices.
> 
> That is strange. Are you sure you don't have a spelling mistake in it. It 
> should be the bowtie2_indices.los I think. I will CC Greg, he is one of the 
> Tool Shed Developers and should know more about it.
> Please do not forget to restart your Galaxy instance once you have update the 
> *loc files.
> 
>> I hope someone on here can help me out.
> 
> Hope it helped a little bit,
> Bjoern
> 
>> Thanks a lot
>> Ravi.
>> ___
>> Please keep all replies on the list by using "reply all"
>> in your mail client.  To manage your subscriptions to this
>> and other Galaxy lists, please use the interface at:
>>   http://lists.bx.psu.edu/
>> 
>> To search Galaxy mailing lists use the unified search at:
>>   http://galaxyproject.org/search/mailinglists/
>> 


___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  http://lists.bx.psu.edu/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/

Re: [galaxy-dev] Graph Datatypes

[Carl Eberhard]

On Tue, Mar 4, 2014 at 6:34 AM, Björn Grüning wrote:

> Hi Carl,
>
> Am 03.03.2014 21:13, schrieb Carl Eberhard:
>
>  Hey, Bjoern
>>
>> Not sure whether a pull request or a toolshed repo would be best, but: if
>> you decide on a toolshed repo, graphview as it is should be able to use
>> your datatypes if they include a 'node-edge' dataprovider.
>>
>
> How can that work? If we migrate all graph-datatypes to the toolshed,
> graphview needs to depend on it, or?
>

I hadn't considered moving the *existing* graph datatypes to the toolshed
as part of the plan, but if we do graphview doesn't necessarily need to be
moved as well (although that would make things more consistent).

The registry and graphview's configuration file just check whether a
dataset has a dataprovider named 'node-edge'. There's no code shared
between them so it's not dependent on the datatype or dataprovider code.

At the very least, moving graphview a seperate decision. Also, it's worth
noting that there currently is no distribution of visualizations from the
tool shed.


>
> Maybe you can comment on my rough plan for that project:
>
> - Graph converter, Tool (every format into every other including json)
> - individual converter to 'auto-convert' every datatype implicitly by
> Galaxy
>  - both are networkx based, so not sure that can be included in Galaxy
> Main, we need at least a new egg
> - write a dataprovider that provides json for every format to graphview
> (omitting the reconstruction of a graph, again in JS)
>

I'd think if you can parse it with networkx, you can pipe that through a
dataprovider to provide the d3 formatted JSON (which is what the
'node-edge' does).


> - enhance the graphview interface a little bit, some control elements,
> ideally an export function to SVG, PNG
> - add filtering to the graphview
>
> I must admit, that I haven't looked much into the details of the JS code
> of graphview but as far as I see its d3.js based and I hope the last two
> points are more or less easy with d3.js
>
>
The last should be fairly easy. The second-to-last is more difficult
(though on my list for the registry as a whole).

Unfortunately, d3 doesn't handle either right now well (mainly due to
browser technology).

The best soln's for SVG seem to involve either a round trip to the server
or asking the user to *right*-click a link a choose 'Save as'. In either
case, including a proper stylesheet for the SVG (or, in the future, a scoped
stylesheet ) becomes an issue -
especially if it will be imported into a PDF, printed, or copied into Word,
etc.

PNG's will involve a round trip to the server and some library on that side
that can reliably and accurately render the SVG onto a bitmap (inkscape or
imageMagick(?)). There's also a small chance that a canvas library could be
used to render the SVG and serve it as a data URI without the round trip.



> Do you have any ToDo items on your list regarding graphview? Should I
> create a Trello Card for it?
>

Not at the moment. Please, feel free and thanks for doing it.

Carl
___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  http://lists.bx.psu.edu/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/

Re: [galaxy-dev] Exporting & Importing Tool Shed repository is just great!

[Peter Cock]

On Tue, Mar 4, 2014 at 2:38 PM, Greg Von Kuster  wrote:
> Hi Peter,
> On Mar 4, 2014, at 5:19 AM, Peter Cock  wrote:
>> On Mon, Mar 3, 2014 at 6:33 PM, Greg Von Kuster  wrote:
 On Mar 3, 2014, at 1:13 PM, Peter Cock  wrote:

 Hi Guys,

 Is this stuff on the wiki?
>>>
>>> Not completely yet.
>>
>> The only mention I found was something in passing on last
>> month's news page:
>> https://wiki.galaxyproject.org/DevNewsBriefs/2014_02_10
>
> Unfortunately I was not able to complete my usual level of documentation
> in the Tool Shed wiki for inclusion in the Galaxy News Brief for the last
> release (my plate was too full at the time).
>
> I've introduced the feature in the following article - this is the venue I
> will use going forward for the Tool Shed.
>
> http://gregvonkuster.org/galaxy-tool-shed-primary-repository-features/

I thought I'd already subscribed to your blog, but evidently hadn't
(or there was a glitch). Done now :)

I would however very much like to see at least a stub entry on the
wiki about the import/export of capsules (linking to the blog) so
it can be found by searching the wiki.

Peter
___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  http://lists.bx.psu.edu/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/

Re: [galaxy-dev] Exporting & Importing Tool Shed repository is just great!

[Greg Von Kuster]

Hi Peter,

On Mar 4, 2014, at 5:19 AM, Peter Cock  wrote:

> On Mon, Mar 3, 2014 at 6:33 PM, Greg Von Kuster  wrote:
>> On Mar 3, 2014, at 1:13 PM, Peter Cock  wrote:
>> 
>>> Hi Guys,
>>> 
>>> Is this stuff on the wiki?
>> 
>> Not completely yet.
> 
> The only mention I found was something in passing on last
> month's news page:
> https://wiki.galaxyproject.org/DevNewsBriefs/2014_02_10

Unfortunately I was not able to complete my usual level of documentation in the 
Tool Shed wiki for inclusion in the Galaxy News Brief for the last release (my 
plate was too full at the time).

I've introduced the feature in the following article - this is the venue I will 
use going forward for the Tool Shed.

http://gregvonkuster.org/galaxy-tool-shed-primary-repository-features/


> 
>>> Presumably the goal is helping to
>>> move repositories between Tool Shed instances (e.g. from
>>> a development/private Tool Shed to a production/public
>>> Tool Shed)?
>> 
>> Yes.
> 
> How do these "capsules" differ from the standard tar-ball?
> Is it effectively the tar-ball contents plus the repository meta
> data like its description and keywords?

From the above article:

Export this revision

This feature is available to all Tool Shed accounts.  It will inspect the 
selected repository revision and create a compressed archive of files that can 
be imported into another Tool Shed.  I’ve called this archive a repository 
capsule.  This feature streamlines the Galaxy utility development process for 
the Tool Shed in that it allows for an entire repository dependency hierarchy 
to easily be moved from one Tool Shed to another.  For example, a repository 
capsule could be exported from a local development Tool Shed and imported into 
the Test Tool Shed hosted by the Galaxy Development Team.  Similarly, a capsule 
could be exported from the Test Tool Shed and imported into the Main Tool Shed. 
 All of the repository’s defined repository dependencies can optionally be 
included in the same capsule.

An XML file named manifest.xml is automatically created and included in the 
capsule.  This file contains the entire list of repositories contained within 
the capsule and the order in which they must be imported into a Tool Shed.  
This file also includes information about each repository (e.g.,the repository 
owner and revision) as well as the Tool Shed categories associated with each of 
them.  The Tool Shed into which the capsule is imported will be inspected to 
see if any of the contained repositories already exists.  Those that do will 
not be overwritten or altered in any way.  A repository created in a Tool Shed 
from an imported capsule will be defined as installable only if its creation 
resulted in no errors.

Since repositories that were exported into a capsule are associated with a user 
(the owner), the user importing the capsule into a Tool Shed must be authorized 
to create the repository in that Tool Shed with that specific owner.  If the 
current user is an admin user or is a member of the Tool Shed’s Intergalactic 
Utilities Commission, all repositories will be created no matter the owner.  
Otherwise, only repositories whose associated owner is the current user will be 
created.

> 
> If so, would this mean a capsule can only be used one at import
> (but cannot be used to update a Tool Shed repository with a
> newer version of the source repository)?

Yes, currently the export / import features supports only the initial import 
into a different tool shed.  The feature has been introduced to streamline the 
Galaxy utility development process by providing an easier way to migrate a set 
of repositories from tool shed to tool shed.  Future enhancements to this 
feature will support updates to repositories.

> 
>>> I can see "Import repository capsule" as a new action at the
>>> bottom of the left hand column (under "Create new repository").
>>> 
>>> Is there an "Export repository capsule" entry somewhere?
>>> I can see a "Export this revision" action on each repository,
>>> is that it?
>>> 
>> 
>> Yes.
> 
> Would it make sense to rename it to "Export capsule for this revision"
> or something more explicit with the word "capsule" in the name?
> (I'm not sure how long these action names can be, but this is a
> little change which would help link the action to the corresponding
> ""Import repository capsule" menu.)

My current feature labels link the actions via the verbs ( export -> import ).  
The act of exporting a repository revision creates a capsule which can then be 
imported.  If more text is needed in the export label, the proper text should 
probably be: "Export this revision into a capsule".  I can certainly make that 
change if there is strong support for it.


> 
> Regards,
> 
> Peter
> ___
> Please keep all replies on the list by using "reply all"
> in your mail client.  To manage your subscriptions to this
> and other Galaxy lists, please use the interface a

Re: [galaxy-dev] Torque.py Bug

[John Chilton]

Great, thanks for testing. This fix has been merged into stable and
default branches of galaxy-central.

-John

On Mon, Mar 3, 2014 at 7:58 PM, ngsf...@hygenomics.com
 wrote:
> hi,John :
> It's ok.
>
> 
> ngsf...@hygenomics.com
>
> From: John Chilton
> Date: 2014-02-28 22:36
> To: ngsf...@hygenomics.com
> CC: galaxy-dev
> Subject: Re: [galaxy-dev] Torque.py Bug
> Thanks for the bug report, that is indeed a problem - I am not sure
> how the torque runner here would ever reach a complete job state. I
> prefer the following fix however, if you would be willing to verify it
> fixes the problem you are encountering I would be happy to apply it to
> Galaxy's stable branch.
>
> diff -r 6a633b37c4c4 lib/galaxy/jobs/runners/cli_job/torque.py
> --- a/lib/galaxy/jobs/runners/cli_job/torque.py Fri Feb 28 08:52:52 2014
> -0500
> +++ b/lib/galaxy/jobs/runners/cli_job/torque.py Fri Feb 28 08:32:41 2014
> -0600
> @@ -131,4 +131,5 @@
>  def __get_job_state(self, state):
>  return { 'E' : job_states.RUNNING,
>   'R' : job_states.RUNNING,
> - 'Q' : job_states.QUEUED }.get(state, state)
> + 'Q' : job_states.QUEUED,
> + 'C' : job_states.OK }.get(state, state)
>
> -John
>
>
> On Fri, Feb 28, 2014 at 12:13 AM, ngsf...@hygenomics.com
>  wrote:
>> Hi,
>>
>> base:
>> os:centos6.4
>> torque:4.2.7
>>
>> I built a glaxy cluster today,and then I found the job completion status
>> of
>> torque queue is 'c',but the need state of galalxy is "ok".So I modified
>> the
>> torque.py file and solved this error. I do, right?
>>
>>
>> diff -r 8411a9f30feb lib/galaxy/jobs/runners/cli_job/torque.py
>> --- a/lib/galaxy/jobs/runners/cli_job/torque.py Thu Dec 19 14:38:03 2013
>> -0500
>> +++ b/lib/galaxy/jobs/runners/cli_job/torque.py Fri Feb 28 13:02:48 2014
>> +0800
>> @@ -117,6 +117,8 @@
>>  if id in job_ids:
>>  state = job.find('job_state').text
>>  # map PBS job states to Galaxy job states.
>> +if state == u'C' :
>> +state = 'ok'
>>  rval[id] = self.__get_job_state(state)
>>  return rval
>>
>> 
>> ngsf...@hygenomics.com
>>
>> ___
>> Please keep all replies on the list by using "reply all"
>> in your mail client.  To manage your subscriptions to this
>> and other Galaxy lists, please use the interface at:
>>   http://lists.bx.psu.edu/
>>
>> To search Galaxy mailing lists use the unified search at:
>>   http://galaxyproject.org/search/mailinglists/
___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  http://lists.bx.psu.edu/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/

Re: [galaxy-dev] Pick-you-own columns in BLAST+ tabular output

[Peter Cock]

On Tue, Mar 4, 2014 at 10:35 AM, Björn Grüning
 wrote:
> Hi Peter,
>
>> Hello all,
>>
>> I've mentioned before that a forthcoming update to the BLAST+
>> Galaxy wrappers would be adding a most customisable output
>> option with pick-your-own columns. This is available to test now
>> from the Test Tool Shed, along with other enhancements like
>> more masking tools and an update to BLAST+ 2.2.29:
>>
>> http://testtoolshed.g2.bx.psu.edu/view/peterjc/ncbi_blast_plus
>>
>> ...
>>
>> Note that this does *not* allow you pick the column order - they
>>
>> will be produced in the order shown. The underlying BLAST+
>> tools would support this, but the Galaxy GUI would I feel be
>> too complicated.
>
>
> I think that is fine, I can't imagine of any real use case where
> that is needed.
>
>>
>> Some community testing/comments would be appreciated.
>
> I tested it briefly on my test instance and it looks all fine and
> seems to work :)
>
> Thanks, looking forward to see it in MainTS.
> Bjoern

Great - thanks. I'm about to update our local production
server, and barring any problems will push this BLAST+
update to the main Tool Shed later this week.

Thanks,

Peter

___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  http://lists.bx.psu.edu/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/

Re: [galaxy-dev] Graph Datatypes

[Björn Grüning]

Hi Carl,

Am 03.03.2014 21:13, schrieb Carl Eberhard:

Hey, Bjoern

Not sure whether a pull request or a toolshed repo would be best, but: if
you decide on a toolshed repo, graphview as it is should be able to use
your datatypes if they include a 'node-edge' dataprovider.


How can that work? If we migrate all graph-datatypes to the toolshed, 
graphview needs to depend on it, or?


Maybe you can comment on my rough plan for that project:

- Graph converter, Tool (every format into every other including json)
- individual converter to 'auto-convert' every datatype implicitly by Galaxy
 - both are networkx based, so not sure that can be included in Galaxy 
Main, we need at least a new egg
- write a dataprovider that provides json for every format to graphview 
(omitting the reconstruction of a graph, again in JS)
- enhance the graphview interface a little bit, some control elements, 
ideally an export function to SVG, PNG

- add filtering to the graphview

I must admit, that I haven’t looked much into the details of the JS code 
of graphview but as far as I see its d3.js based and I hope the last two 
points are more or less easy with d3.js


Do you have any ToDo items on your list regarding graphview? Should I 
create a Trello Card for it?


Thanks,
Bjoern


Also, graphview is on my list - but most likely I won't get to it in the
near future.

Let me know if I can help with any of your changes,
Carl



On Mon, Mar 3, 2014 at 5:09 AM, Björn Grüning wrote:


Hi,

we are currently working on a few graph based tools and we would like to
add some new graph-datatypes, namely: gspan, gml, gexf, graphml, pajek. I
saw that xgmml and sif are already integrated into core-galaxy and I'm
wondering if a PR with these new datatypes would be appropriate or a new
toolshed repository?

Also, are there any plans to enhance graphview in the near future?

Thanks,
Bjoern




___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
 http://lists.bx.psu.edu/

To search Galaxy mailing lists use the unified search at:
 http://galaxyproject.org/search/mailinglists/

Re: [galaxy-dev] How to install bowtie2 tool in galaxy

[Björn Grüning]

Hi Ravi,


Hi guys,
I am new to galaxy and am in the process of setting it up on a cluster with 
some help. I am trying to install bowtie2 to my local galaxy through the 
toolshed. I have a few questions.

- Which bowtie2 should I install? When I search for bowtie2 a bunch of results 
come up?


Only two, or? Please make sure you are using the main toolshed. bowtie2 
and package_bowtie2_2_1_0 are connected together. One is the binary and 
the other contains the wrapper. Use devteam respositories is always a 
good choice.



- What are repository dependencies? Why would I need these if bowtie2 is 
already installed on my system?


repository dependecies containing all dependencies of your wrappers, 
that can be some binaries, but also R, Perl, python libraries that are 
needed to execute your wrappers. We recommend you to use the 
dependencies because then you have full control over all your used 
versions (wrapper, dependencies, tool) and that enables reproducibility. 
To put in other words: With the toolshed you can have different tool and 
wrappers versions with different dependencies at the same time.



- I have been installing bowtie2 by unchecking the repository dependencies and 
tool dependencies boxes. Is this correct?


It will work, if you have a system installed version. But you need to 
take care about binary updates by your own. And reproducibility of your 
results is not guaranteed.



- Finally when I install bowtie2 this way it creates 2 bowtie2_indices.loc 
files, one in galaxy/tool-data and the other in 
shed_tools/toolshed.g2.gx.psu.edu/dev/bowtie2, which one do I need to edit to 
point to my index files? No matter which one I change I can't seem to see the 
indices.


That is strange. Are you sure you don't have a spelling mistake in it. 
It should be the bowtie2_indices.los I think. I will CC Greg, he is one 
of the Tool Shed Developers and should know more about it.
Please do not forget to restart your Galaxy instance once you have 
update the *loc files.



I hope someone on here can help me out.


Hope it helped a little bit,
Bjoern


Thanks a lot
Ravi.
___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
   http://lists.bx.psu.edu/

To search Galaxy mailing lists use the unified search at:
   http://galaxyproject.org/search/mailinglists/


___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
 http://lists.bx.psu.edu/

To search Galaxy mailing lists use the unified search at:
 http://galaxyproject.org/search/mailinglists/

Re: [galaxy-dev] Pick-you-own columns in BLAST+ tabular output

[Björn Grüning]

Hi Peter,


Hello all,

I've mentioned before that a forthcoming update to the BLAST+
Galaxy wrappers would be adding a most customisable output
option with pick-your-own columns. This is available to test now
from the Test Tool Shed, along with other enhancements like
more masking tools and an update to BLAST+ 2.2.29:

http://testtoolshed.g2.bx.psu.edu/view/peterjc/ncbi_blast_plus

(This will be released as the ncbi_blast_plus wrappers v0.1.0,
and includes recent contributions from Jim Johnson and
Björn Grüning - thank you both):

Previously (for tabular output) we've only offered the standard
12 column output (-outfmt 6), and an extended set which has
grown to currently include 25 columns.

The new custom column mode offers the 12 standard columns,
and the extended set (making 25 columns), plus other (to me
less interesting) identifiers, some other miscellaneous
columns, and the taxonomy columns (44 columns in all):

[image: Inline image 1]
[image: Inline image 2]

As previously discussed, the list of columns has been grouped
- the first two groups reflect the standard 12 columns and the
columns used in Galaxy's (default) 25 column extended tabular
output.

The remaining columns I have split into three groups (assorted
identifiers, miscellaneous, and taxonomy) which I hope is self
explanatory.

Note that this does *not* allow you pick the column order - they
will be produced in the order shown. The underlying BLAST+
tools would support this, but the Galaxy GUI would I feel be
too complicated.


I think that is fine, I can't imagine of any real use case where that is 
needed.



Some community testing/comments would be appreciated.


I tested it briefly on my test instance and it looks all fine and seems 
to work :)


Thanks, looking forward to see it in MainTS.
Bjoern



Thanks,

Peter


___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
 http://lists.bx.psu.edu/

To search Galaxy mailing lists use the unified search at:
 http://galaxyproject.org/search/mailinglists/

Re: [galaxy-dev] Exporting & Importing Tool Shed repository is just great!

[Peter Cock]

On Mon, Mar 3, 2014 at 6:33 PM, Greg Von Kuster  wrote:
> On Mar 3, 2014, at 1:13 PM, Peter Cock  wrote:
>
>> Hi Guys,
>>
>> Is this stuff on the wiki?
>
> Not completely yet.

The only mention I found was something in passing on last
month's news page:
https://wiki.galaxyproject.org/DevNewsBriefs/2014_02_10

>> Presumably the goal is helping to
>> move repositories between Tool Shed instances (e.g. from
>> a development/private Tool Shed to a production/public
>> Tool Shed)?
>
> Yes.

How do these "capsules" differ from the standard tar-ball?
Is it effectively the tar-ball contents plus the repository meta
data like its description and keywords?

If so, would this mean a capsule can only be used one at import
(but cannot be used to update a Tool Shed repository with a
newer version of the source repository)?

>> I can see "Import repository capsule" as a new action at the
>> bottom of the left hand column (under "Create new repository").
>>
>> Is there an "Export repository capsule" entry somewhere?
>> I can see a "Export this revision" action on each repository,
>> is that it?
>>
>
> Yes.

Would it make sense to rename it to "Export capsule for this revision"
or something more explicit with the word "capsule" in the name?
(I'm not sure how long these action names can be, but this is a
little change which would help link the action to the corresponding
""Import repository capsule" menu.)

Regards,

Peter
___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  http://lists.bx.psu.edu/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/