[galaxy-dev] tophat xml that supports SOLID colorspace

[KOH Jia Yu Jayce]

Hi,

I am looking for tophat xml that supports SOLID colorspace. Is it anywhere 
available?

Thanks a lot.
___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:

  http://lists.bx.psu.edu/

[galaxy-dev] How to allow anonymous users to run workflows?

[Tim te Beek]

Hi all,

Was wondering how I can allow anonymous users to run workflows in my
local Galaxy instance, as currently users need to be logged in to run
workflows. I'd like drop this requirement in light of the intended
publication of a workflow in a journal which demands that "Web
services must not require mandatory registration by the user.". Could
any you tell me how I can accomplish this?

I've seen the option to use an external authentication method which
could be employed to artificially 'login' anonymous users for a single
session, but it appears this would also disable the normal users
administration mechanisms in Galaxy, so I'm not sure this would be a
good fit. Any hints on how to proceed, either via this route or
otherwise, would be much appreciated.

Best regards,
Tim
___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:

  http://lists.bx.psu.edu/

[galaxy-dev] Path to tool designation in tool config xml

[Timothy Wu]

Hi,

I'm trying the add tool tutorial at
http://wiki.g2.bx.psu.edu/Admin/Tools/Add%20Tool%20Tutorial . It works, but
how do I designate the path of the tool if it's not dropped into the same
directory as the tool config xml? Some executable may not be in the
directory. Surely it couldn't possiblely be the case with external tools
like BLAST, isn't it?! But looking at the BLAST config xmls I couldn't make
out the difference and I didn't find any documentation on how to designate
the path of the tool. Help appreciated, thank you.

Timothy
___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:

  http://lists.bx.psu.edu/

Re: [galaxy-dev] How to allow anonymous users to run workflows?

[Ross]

Here's one bad option for dealing with a bad requirement - at least it
requires no code changes...

1. create a new user called d...@where.ever.org for your Galaxy. Do
not require login in universe_wsgi.ini
2. edit welcome.html - invite visitors not wanting all the benefits of
an individual registration to login as user:d...@where.ever.org if
they want to run workflows
3. Add an extremely blunt disclaimer - although Galaxy will likely be
behaving 'correctly', unusual things will happen whenever 2 or more
users using the same account are banging away at the same history -
Individual registration is strongly recommended - register a free
account with a throw away email address - it's only for password
recovery.
Clearly registration is not mandatory - just sensible and free, so the
journal's off your back. Hope that demo account isn't multitasking too
often.

On Tue, Sep 6, 2011 at 6:17 PM, Tim te Beek  wrote:
> Hi all,
>
> Was wondering how I can allow anonymous users to run workflows in my
> local Galaxy instance, as currently users need to be logged in to run
> workflows. I'd like drop this requirement in light of the intended
> publication of a workflow in a journal which demands that "Web
> services must not require mandatory registration by the user.". Could
> any you tell me how I can accomplish this?
>
> I've seen the option to use an external authentication method which
> could be employed to artificially 'login' anonymous users for a single
> session, but it appears this would also disable the normal users
> administration mechanisms in Galaxy, so I'm not sure this would be a
> good fit. Any hints on how to proceed, either via this route or
> otherwise, would be much appreciated.
>
> Best regards,
> Tim
___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:

  http://lists.bx.psu.edu/

Re: [galaxy-dev] Path to tool designation in tool config xml

[Peter Cock]

On Tue, Sep 6, 2011 at 9:41 AM, Timothy Wu <2hug...@gmail.com> wrote:
> Hi,
>
> I'm trying the add tool tutorial at
> http://wiki.g2.bx.psu.edu/Admin/Tools/Add%20Tool%20Tutorial . It works, but
> how do I designate the path of the tool if it's not dropped into the same
> directory as the tool config xml? Some executable may not be in the
> directory. Surely it couldn't possiblely be the case with external tools
> like BLAST, isn't it?! But looking at the BLAST config xmls I couldn't make
> out the difference and I didn't find any documentation on how to designate
> the path of the tool. Help appreciated, thank you.
>
> Timothy

Normally on a Unix/Linux system you'd just put the tool on the path
(or add the tool's location to the system path).

Peter
___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:

  http://lists.bx.psu.edu/

[galaxy-dev] Regarding display application

[shashi shekhar]

Hi All.,

 I need help in understanding the xml tag for display_application. i
don't know how all these xml tags works . can u provide me links for
understanding those tags like

 target_frame

type="template"

viewable="False




http://genetrack.g2.bx.psu.edu/galaxy?filename=${encoded_filename.qp}&hashkey=${hash_key.qp}&input=${qp(str($genetrack_file.id))}&GALAXY_URL=${galaxy_url.qp}
 


${BASE_URL}/tool_runner?tool_id=predict2genetrack


#from galaxy.util.hash_util import hmac_new
${hmac_new( $APP.config.tool_secret, $genetrack_file.file_name )}


#import binascii
${binascii.hexlify( $genetrack_file.file_name )}








Regards
shashi shekhar
___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:

  http://lists.bx.psu.edu/

Re: [galaxy-dev] Regarding display application

[shashi shekhar]

On Tue, Sep 6, 2011 at 3:33 PM, shashi shekhar  wrote:
> Hi All.,
>
>  I need help in understanding the xml tag for display_application. i
> don't know how all these xml tags works . can u provide me links for
> understanding those tags like
>
>  target_frame
>
> type="template"
>
> viewable="False
>
>
> 
>    
>         target_frame="galaxy_main">http://genetrack.g2.bx.psu.edu/galaxy?filename=${encoded_filename.qp}&hashkey=${hash_key.qp}&input=${qp(str($genetrack_file.id))}&GALAXY_URL=${galaxy_url.qp}
>         format="bed6,genetrack"/> 
>         format="genetrack" viewable="False" />
>        
>            ${BASE_URL}/tool_runner?tool_id=predict2genetrack
>        
>        
>            #from galaxy.util.hash_util import hmac_new
>            ${hmac_new( $APP.config.tool_secret, $genetrack_file.file_name )}
>        
>        
>            #import binascii
>            ${binascii.hexlify( $genetrack_file.file_name )}
>        
>    
> 
>
>
>
>
>
> Regards
> shashi shekhar
>

___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:

  http://lists.bx.psu.edu/

Re: [galaxy-dev] Regarding display application

[Daniel Blankenberg]

Hi Shashi,

The docs for external display application could use some enhancement, but I 
have provided details on the features you are asking about below.

target_frame: this is the frame where you want this external display 
application to appear.

type="template": this parameter type is a cheetah template; like  and 
 tags in tools 

viewable="False": is this parameter viewable over http/s? When set to false, 
you will be able to access and use the parameter in the generation of the 
display applications, but it will not be accessible over the web via a URL. 
GeneTrack accesses files directly over the filesystem, so the files do not need 
to be web accessible for the display application.


Thanks for using Galaxy,

Dan



On Sep 6, 2011, at 6:03 AM, shashi shekhar wrote:

> Hi All.,
> 
> I need help in understanding the xml tag for display_application. i
> don't know how all these xml tags works . can u provide me links for
> understanding those tags like
> 
> target_frame
> 
> type="template"
> 
> viewable="False
> 
> 
> 
>
> target_frame="galaxy_main">http://genetrack.g2.bx.psu.edu/galaxy?filename=${encoded_filename.qp}&hashkey=${hash_key.qp}&input=${qp(str($genetrack_file.id))}&GALAXY_URL=${galaxy_url.qp}
> format="bed6,genetrack"/> 
> format="genetrack" viewable="False" />
>
>${BASE_URL}/tool_runner?tool_id=predict2genetrack
>
>
>#from galaxy.util.hash_util import hmac_new
>${hmac_new( $APP.config.tool_secret, $genetrack_file.file_name )}
>
>
>#import binascii
>${binascii.hexlify( $genetrack_file.file_name )}
>
>
> 
> 
> 
> 
> 
> 
> Regards
> shashi shekhar
> ___
> Please keep all replies on the list by using "reply all"
> in your mail client.  To manage your subscriptions to this
> and other Galaxy lists, please use the interface at:
> 
>  http://lists.bx.psu.edu/


___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:

  http://lists.bx.psu.edu/

[galaxy-dev] Quotas

[SHAUN WEBB]

Hi.

I am interested in putting quotas in to effect on our local Galaxy  
server. From the admin page I can create a quota but this only seems  
to apply to disk space. I noticed on your test site you have quotas  
for concurrent jobs also. Have I missed how to set this or is this  
functionality not available in galaxy-dist yet?


2 other issues:

1) My account says I am using "0 bytes" in the top right corner  
although my history displays 456mb of data?


2) Are quotas set by default? I was able to create quotas before I had  
updated universe_wsgi.ini to include enable_quotas = True



Shaun

--
The University of Edinburgh is a charitable body, registered in
Scotland, with registration number SC005336.


___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:

 http://lists.bx.psu.edu/

Re: [galaxy-dev] TEMP variable in cluster install

[Nate Coraor (n...@bx.psu.edu)]

Chorny, Ilya wrote:
> Nate,
> 
> We ran into this issue with  /tmp not having enough space. We came up with a 
> generic solution and were wondering if you might want to add it to 
> galaxy-central. We did not want to modify SGE as it could affect other jobs 
> that use the cluster. 
> 
> We created a TMPDIR  option in universe_wsgi.ini and then modified drama.py 
> to include an "export TMPDIR=TMPDIR" to the drm_tmplate if TMPDIR is defined 
> in universe_wsgi.ini.  Do you want to pull this code into galaxy central. 
> It's like 5 lines of code. 

This is also something you can put in the Galaxy user's ~/.sge_request,
although with your actual user code this would be more difficult.

There's actually already a temp directory option, new_file_path, this
would just need to be added to the template.  But I don't know that it's
a good idea, since each node may have a large local scratch space and
the path to that space may differ (this is the case in our environment,
where we actually run jobs on multiple clusters).

> 
>  Thanks and have a good weekend,
> 
> Ilya
> 
> BTW, any luck reviewing the run as actual user code?

I should be doing it today.

--nate

> 
> 
> -Original Message-
> From: galaxy-dev-boun...@lists.bx.psu.edu 
> [mailto:galaxy-dev-boun...@lists.bx.psu.edu] On Behalf Of Assaf Gordon
> Sent: Thursday, August 04, 2011 2:31 PM
> To: Shantanu Pavgi; galaxydev psu
> Subject: Re: [galaxy-dev] TEMP variable in cluster install
> 
> As a follow-up for SGE + TMP variables,
> 
> I've also encountered problems with settings TEMP,TMP and TMPDIR variables.
> Tried setting "~/.sge_request" and changing the Galaxy environment variables 
> before starting python - nothing worked - TMPDIR and TMP were always set to 
> "/tmp/all.q.X" by SGE .
> 
> What finally worked is changing the SGE configuration queue, and simply 
> setting the "tmpdir" variable to my desired temp directory.
> 
> Do that by running:
> $ sudo qconf -mq all.q
> 
> and changing the line:
>   tmpdir/tmp
> to
>   tmpdir/my/temporary/directory/path
> 
> 
> Problem solved :)
> No more messing around with TMP,TMPDIR variables in any Galaxy related source 
> files.
> 
> Hope this helps,
>  -gordon
> 
> 
> 
> 
> Shantanu Pavgi wrote, On 07/26/2011 05:40 PM:
> > 
> > On Jul 21, 2011, at 6:24 PM, Shantanu Pavgi wrote:
> > 
> >>
> >> We have configured galaxy to work with our SGE cluster using drmaa 
> >> job runner interface. We are using 'unified method' for this install and 
> >> both TEMP environment variable and new_file_path in universe_wsgi.ini file 
> >> have been configured correctly. However, we are seeing some errors where 
> >> local /tmp space on compute nodes is being referenced by the galaxy tools. 
> >> Specifically we saw it mentioned in error messages from following tools:
> >> * bwa_wrapper and upload tools: 'No space left on device: /tmp...' 
> >> * sam_to_bam tool: 'Error sorting alignments from /tmp/filename..'
> >>
> >> Shouldn't it be referencing TEMP environment variable or new_file_path 
> >> configuration value? Is it getting overridden by TMP or TMPDIR variables 
> >> in python wrapper code? Has anyone else experienced similar issue? 
> >>
> >>
> > 
> > Further debugging showed that there are two more temporary directory 
> > related environment variables - TMP and TMPDIR which were pointing local 
> > /tmp location on compute nodes. We tried to set these variables in our 
> > shell environment (drmaa URL uses -V to export current shell env) however 
> > SGE overwrote TMP and TMPDIR before actual job execution. The TEMP variable 
> > remained unmodified by SGE scheduler.
> > 
> > The galaxy tools seemed to be using temporary directory space pointed by 
> > TMP and TMPDIR and hence we saw local /tmp related errors mentioned in my 
> > earlier post. 
> > 
> > We have temporarily fixed this problem by hard coding TEMP, TMP and 
> > TMPDIR values in job template files - 1. 
> > lib/galaxy/jobs/runners/drmaa.py 2. lib/galaxy/jobs/runners/sge.py
> > 
> > This will affect all jobs submitted through the galaxy. It will be helpful 
> > to know how it can be handled in the tool wrapper scripts. It seems like it 
> > can be handled in the tool wrapper scripts using directory prefix while 
> > creating the tempfile ( http://docs.python.org/library/tempfile.html ). Any 
> > thoughts/comments?  Are there any other SGE users having similar issue? 
> > 
> > Also, there is a similar thread started today regarding usage of temporary 
> > directory location in tool wrapper scripts. It would be helpful to know how 
> > galaxy or tool wrappers are using the temporary directory and ways to 
> > configure it. 
> > 
> > --
> > Thanks,
> > Shantanu. 
> > ___
> > Please keep all replies on the list by using "reply all"
> > in your mail client.  To manage your subscriptions to this and other 
> > Galaxy lists, please use the interface at:
> > 
> >   http://lists.bx.psu.edu/
>

Re: [galaxy-dev] Regarding display application

[shashi shekhar]

Hi all,

  Thanks for reply .   I have an  application  to visualize some
custom datatyes inputs . i want to integrate this application  with
galaxy . i have already added datatypes in my local instance of
Galaxy . Is It  required  to modify my  application  ?


Regards
shashi


On Tue, Sep 6, 2011 at 6:42 PM, Daniel Blankenberg  wrote:
> Hi Shashi,
>
> The docs for external display application could use some enhancement, but I 
> have provided details on the features you are asking about below.
>
> target_frame: this is the frame where you want this external display 
> application to appear.
>
> type="template": this parameter type is a cheetah template; like  
> and  tags in tools
>
> viewable="False": is this parameter viewable over http/s? When set to false, 
> you will be able to access and use the parameter in the generation of the 
> display applications, but it will not be accessible over the web via a URL. 
> GeneTrack accesses files directly over the filesystem, so the files do not 
> need to be web accessible for the display application.
>
>
> Thanks for using Galaxy,
>
> Dan
>
>
>
> On Sep 6, 2011, at 6:03 AM, shashi shekhar wrote:
>
>> Hi All.,
>>
>> I need help in understanding the xml tag for display_application. i
>> don't know how all these xml tags works . can u provide me links for
>> understanding those tags like
>>
>> target_frame
>>
>> type="template"
>>
>> viewable="False
>>
>>
>> 
>>    
>>        > target_frame="galaxy_main">http://genetrack.g2.bx.psu.edu/galaxy?filename=${encoded_filename.qp}&hashkey=${hash_key.qp}&input=${qp(str($genetrack_file.id))}&GALAXY_URL=${galaxy_url.qp}
>>        > format="bed6,genetrack"/> 
>>        > format="genetrack" viewable="False" />
>>        
>>            ${BASE_URL}/tool_runner?tool_id=predict2genetrack
>>        
>>        
>>            #from galaxy.util.hash_util import hmac_new
>>            ${hmac_new( $APP.config.tool_secret, $genetrack_file.file_name )}
>>        
>>        
>>            #import binascii
>>            ${binascii.hexlify( $genetrack_file.file_name )}
>>        
>>    
>> 
>>
>>
>>
>>
>>
>> Regards
>> shashi shekhar
>> ___
>> Please keep all replies on the list by using "reply all"
>> in your mail client.  To manage your subscriptions to this
>> and other Galaxy lists, please use the interface at:
>>
>>  http://lists.bx.psu.edu/
>
>

___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:

  http://lists.bx.psu.edu/

Re: [galaxy-dev] disk space and file formats

[Nate Coraor]

Edward Kirton wrote:
> > In your position I agree that is a pragmatic choice.
> 
> Thanks for helping me muddle through my options.
> 
> > You might be able to
> > modify the file upload code to gzip any FASTQ files... that would prevent
> > uncompressed FASTQ getting into new histories.
> 
> Right!
> 
> > I wonder if Galaxy would benefit from a new fastqsanger-gzip (etc) datatype?
> > However this seems generally useful (not just for FASTQ) so perhaps a more
> > general mechanism would be better where tool XML files can say which file
> > types they accept and which of those can/must be compressed (possily not
> > just gzip format?).
> 
> Perhaps we can flesh-out what more general solutions would look like...
> 
> Imagine the fastq datatypes were left alone and instead there's a
> mechanism by which files which haven't been used as input for x days
> get compressed by a cron job.  the file server knows how to uncompress
> such files on the fly when needed.  For the most part, files are
> uncompressed during analysis and are compressed when the files exist
> as an archive within galaxy.

Ideally, there'd just be a column on the dataset table indicating
whether the dataset is compressed or not, and then tools get a new
way to indicate whether they can directly read compressed inputs, or
whether the input needs to be decompressed first.

--nate

> 
> An even simpler solution would be an archive/compress button which
> users could use when they're done with a history.  Users could still
> copy (uncompressed) datasets into a new history for further analysis.
> 
> Of course there's also the solution mentioned in the 2010 galaxy
> developer's conference about automatic compression at the system
> level.  Not a possibility for me, but is attractive.
> ___
> Please keep all replies on the list by using "reply all"
> in your mail client.  To manage your subscriptions to this
> and other Galaxy lists, please use the interface at:
> 
>   http://lists.bx.psu.edu/
___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:

  http://lists.bx.psu.edu/

Re: [galaxy-dev] disk space and file formats

[Peter Cock]

On Tue, Sep 6, 2011 at 3:24 PM, Nate Coraor  wrote:
> Edward Kirton wrote:
>> Peter wrote:
>> > I wonder if Galaxy would benefit from a new fastqsanger-gzip (etc) 
>> > datatype?
>> > However this seems generally useful (not just for FASTQ) so perhaps a more
>> > general mechanism would be better where tool XML files can say which file
>> > types they accept and which of those can/must be compressed (possily not
>> > just gzip format?).
>>
>> Perhaps we can flesh-out what more general solutions would look like...
>>
>> Imagine the fastq datatypes were left alone and instead there's a
>> mechanism by which files which haven't been used as input for x days
>> get compressed by a cron job.  the file server knows how to uncompress
>> such files on the fly when needed.  For the most part, files are
>> uncompressed during analysis and are compressed when the files exist
>> as an archive within galaxy.
>
> Ideally, there'd just be a column on the dataset table indicating
> whether the dataset is compressed or not, and then tools get a new
> way to indicate whether they can directly read compressed inputs, or
> whether the input needs to be decompressed first.
>
> --nate

Yes, that's what I was envisioning Nate.

Are there any schemes other than gzip which would make sense?
Perhaps rather than a boolean column (compressed or not), it
should specify the kind of compression if any (e.g. gzip).

We need something which balances compression efficiency (size)
with decompression speed, while also being widely supported in
libraries for maximum tool uptake.

Peter

___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:

  http://lists.bx.psu.edu/

Re: [galaxy-dev] rpy - No module named rpy CentoOs install

[Nate Coraor]

Joseph Hargitai wrote:
> 
> Nate, 
> 
> could we go to the beginning of the issue:
> 
> where is the galaxy env set?  I've seen a few post but I can only gather 
> partial info.

When running locally, the environment is the environment that is in
place when you start Galaxy.  If you're starting via an init script,
this is probably not the environment of your current shell, however.

When running via SGE, the environment is the environment that is set up
when the shell starts on the execution host.  SGE usually starts in
non-login, non-interactive mode, and this means that it reads almost
nothing.  In this case, the best way to set up the environment is in
~/.sge_request.

See the INVOCATION section of the bash(1) man page for further
clarification.

> - it is NOT set from the galaxy user .bashrc or .profile

If starting from an interactive login shell, then .profile would be read
by bash at startup.  .bashrc is read if bash is started as an
interactive non-login shell, although many installations have a .profile
or .bash_profile set up to read .bashrc to negate this limatation.

> - if it is indeed partially set from /etc/profile - using a Rocks cluster 
> leaves you with many entries there to ponder

This is probably not feasible since it requires system-wide changes on
every node.

> - if it is using ld.so.conf.d as well - it will read /usr/lib64 entries 
> etc... 

This does not set up environment variables or Python's sys.path, only
search paths for the runtime linker.

> - is there a precise way to see what env is used for galaxy? The log script 
> gives you a nice read on the python path but is there a way to see all envs? 
> Looking at envs as the user "galaxy" does not equate what galaxy ends up 
> using.

Sure, create a tool that runs `set` and/or `printenv`.

> multiple issues on the CentOS install:
> 
> I found the setting or non-setting  leading to the missing rpy module by 
> looking at the runner log script - 
> while it was loading python2.6.6 it was also loading the site-packages and 
> other python parts from /usr/lib64...python2.4 
> 
> Once I edited run.sh to use the correct python and correct R path and added 
> the RHOME to the rpy dependent scripts - this problem went away seemingly 
> only to produce an env looking issue: sh rm command not found when running 
> rpy dependent applications. Did somehow the edit destroy the /bin and usr/bin 
> path? Would these be set in run.sh as well?   

This would indicate that $PATH was set to something like:

PATH=/foo

rather than:

PATH=/foo:$PATH

> To your question: 
> where do you set RHOME in the env? 
> 
> We'd prefer to set all path options in run.sh in case all above is true that 
> you cannot set it in ~/.bash* 

>From Galaxy's standpoint, avoiding changes to run.sh is also preferable
since it is version controlled (although you could make your own copy
with modifications).  Also, as outlined above, changes in run.sh are not
propogated to the nodes when running on a cluster.

--nate

> 
> best,
> joe
> 
> From: Nate Coraor [n...@bx.psu.edu]
> Sent: Friday, September 02, 2011 1:40 PM
> To: Joseph Hargitai
> Cc: galaxy-dev@lists.bx.psu.edu
> Subject: Re: [galaxy-dev] rpy - No module named rpy CentoOs install
> 
> Joseph Hargitai wrote:
> > additional info:
> >
> > it is possible on the same node to run manually
> >
> > ./gsummary.py
> >
> > with the header:
> >
> > #!/usr/bin/env python
> >
> > import sys, re, tempfile
> > from rpy_options import set_options
> > set_options(RHOME='/apps1/R/2.13.1/intel/lib64/R')
> > from rpy import *
> >
> > Where else can there be an env setting to prevent this app not finding the 
> > mod from within galaxy?
> 
> Hi Joe,
> 
> If you set RHOME in the environment and then run gsummary.py without the
> additions, does it work?
> 
> --nate
> 
> >
> > j
> >
> >
> > 
> > From: Joseph Hargitai
> > Sent: Thursday, September 01, 2011 12:28 PM
> > To: galaxy-dev@lists.bx.psu.edu
> > Subject: rpy - No module named rpy CentoOs install
> >
> > Hi,
> >
> > On our Ubuntu install stat packages and all that require rpy work fine.
> >
> > On our CentOs install seeing this stubborn error that I did see from 
> > previous post to be difficult to fix.
> >
> > At first suspected the SGE issue - environment not transferring to  compute 
> > nodes. After changing the app to run local had the same issue.
> >
> > CentOs: 2.6.18-92.1.13.el5
> >
> > rpy module is in:
> >
> > /apps1/python/2.6.6/intel/lib/python2.6/site-packages
> >
> > _rpy2122.so
> > _rpy2131.so
> >
> > version:
> > [galaxy@compute-0-65 galaxy-dist]$ python -c "import rpy; print 
> > rpy.__version__"
> > 1.5.1
> >
> > path:
> >
> > python -c 'import sys; print "\n".join( sys.path )'
> >
> > /apps1/python/2.6.6/intel/lib/python2.6/site-packages/simplejson-2.0.9-py2.6-linux-x86_64.egg
> > /apps1/python/2.6.6/intel/lib/python2.6/site-packages/Sphinx-1.0.7-py2.6.egg
> > /apps1/python/2.6.6

Re: [galaxy-dev] How to allow anonymous users to run workflows?

[Tim te Beek]

I completely agree it's a bit of an unfortunate requirement in this
case, but I'm not averse to (minor) code changes to achieve a more
polished user experience. Something like the best guess mechanism
Galaxy currently employs to recognize returning anonymous would be
fine, but I don't know where to look to disable the login requirement
to run workflows, or if that's at all possible.

Best regards,
Tim

On Tue, Sep 6, 2011 at 11:01 AM, Ross  wrote:
> Here's one bad option for dealing with a bad requirement - at least it
> requires no code changes...
>
> 1. create a new user called d...@where.ever.org for your Galaxy. Do
> not require login in universe_wsgi.ini
> 2. edit welcome.html - invite visitors not wanting all the benefits of
> an individual registration to login as user:d...@where.ever.org if
> they want to run workflows
> 3. Add an extremely blunt disclaimer - although Galaxy will likely be
> behaving 'correctly', unusual things will happen whenever 2 or more
> users using the same account are banging away at the same history -
> Individual registration is strongly recommended - register a free
> account with a throw away email address - it's only for password
> recovery.
> Clearly registration is not mandatory - just sensible and free, so the
> journal's off your back. Hope that demo account isn't multitasking too
> often.
>
> On Tue, Sep 6, 2011 at 6:17 PM, Tim te Beek  wrote:
>> Hi all,
>>
>> Was wondering how I can allow anonymous users to run workflows in my
>> local Galaxy instance, as currently users need to be logged in to run
>> workflows. I'd like drop this requirement in light of the intended
>> publication of a workflow in a journal which demands that "Web
>> services must not require mandatory registration by the user.". Could
>> any you tell me how I can accomplish this?
>>
>> I've seen the option to use an external authentication method which
>> could be employed to artificially 'login' anonymous users for a single
>> session, but it appears this would also disable the normal users
>> administration mechanisms in Galaxy, so I'm not sure this would be a
>> good fit. Any hints on how to proceed, either via this route or
>> otherwise, would be much appreciated.
>>
>> Best regards,
>> Tim
>
___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:

  http://lists.bx.psu.edu/

Re: [galaxy-dev] link file bug in the new version

[Nate Coraor]

remy d1 wrote:
> Hello Nate,
> 
> It is not easy...
> 
> I think it is in this :
> 
> if dataset.type in ( 'server_dir', 'path_paste' ) and link_data_only ==
> >> 'link_to_files':
> >
> > # Never alter a file that will not be copied to Galaxy's local file
> >> store.
> >
> > if datatype.dataset_content_needs_grooming( dataset.path ):
> >
> > err_msg = 'The uploaded files need grooming, so change your
> >> Copy data into Galaxy? selection to be ' + \
> >
> > 'Copy files into Galaxy instead of Link to files
> >> without copying into Galaxy so grooming can be performed.'
> >
> > file_err( err_msg, dataset, json_file )
> >
> > return
> >
> >
> In fact, I have to say that the file does not diseappear. The file is still
> existing, but it is* empty* and it is occur only when the owner of the file
> is the same that the galaxy user (the galaxy service is running under galaxy
> user too).

Hi Remy,

I still can't duplicate this.  Is it with a specific file type (and are
you setting the file type explicitly, or using auto-detection)?

> Where can i find the dataset class ?

The datatypes are defined in galaxy-dist/lib/galaxy/datatypes/

--nate

> 
> Perhaps, the operation does not work and there is a link with this stuff :
> 
> http://bytes.com/topic/python/answers/26169-shutil-move-error-errno-17-file-exists-win32
> 
>  - Regards
> 
> 2011/9/2 Nate Coraor 
> 
> > remy d1 wrote:
> > > Hi,
> > >
> > > We found a little problem in the new galaxy release. When we upload a
> > > dataset library from filesystem :
> > >
> > > Admin > Manage data libraries > create new data library > Add dataset >
> > > Upload files from filesystem path > Link to files without copying
> > >
> > > If galaxy user is the owner of this file or if he has write permission on
> > it
> > > (on the filesystem), the file is deleted !!
> > >
> > >
> > > I do not think it is the normal behaviour...
> >
> > Hi Remy,
> >
> > I'm unable to duplicate this behavior.  Would it be possible for you to
> > do some debugging on your end to determine when this is happening?  If
> > it's Galaxy's code it would probably be somewhere in
> > tools/data_source/upload.py
> >
> > --nate
> >
> > >
> > >
> > > Regards.
> >
> > > ___
> > > Please keep all replies on the list by using "reply all"
> > > in your mail client.  To manage your subscriptions to this
> > > and other Galaxy lists, please use the interface at:
> > >
> > >   http://lists.bx.psu.edu/
> >
> >
___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:

  http://lists.bx.psu.edu/

Re: [galaxy-dev] Galaxy egg fetching error? Mac OS X 10.7 (Lion)

[Nate Coraor]

Whyte, Jeffrey wrote:
> Thanks for the information, Nate.
> 
> I installed Python as you described, and the error I get now is pasted below. 
>  I appreciate your help.
> 
> Jeff
> 
> [~/galaxy-dist] myuserid 03:06 PM > ./run.sh
> Some eggs are out of date, attempting to fetch...
> Warning: MarkupSafe (a dependent egg of Mako) cannot be fetched
> Warning: simplejson (a dependent egg of WebHelpers) cannot be fetched
> pysam 0.4.2 couldn't be downloaded automatically.  You can try
> building it by hand with:
>   python scripts/scramble.py -e pysam
> Fetch failed.
> [~/galaxy-dist] myuserid 03:06 PM >

Could you report back with the output of:

% python ./scripts/get_platforms.py

?

Thanks,
--nate

> 
> From: Nate Coraor [n...@bx.psu.edu]
> Sent: Friday, September 02, 2011 1:16 PM
> To: Whyte, Jeffrey
> Cc: galaxy-dev@lists.bx.psu.edu
> Subject: Re: [galaxy-dev] Galaxy egg fetching error? Mac OS X 10.7 (Lion)
> 
> Whyte, Jeffrey wrote:
> > Hi,
> >
> > I've been having trouble running a local instance of Galaxy on a Mac Pro 
> > after upgrading to OS X 10.7 (Lion).  My Python version is 2.7.1 and 
> > Mercurial is 1.9.1 for MacOS X 10.7.  I don't have any MacPorts installed.
> >
> > The error I see after running the startup script is pasted at the end of 
> > this message.  Thanks in advance for any help or advice.
> > jjw
> 
> Hi Jeffrey,
> 
> We haven't yet gotten our dependencies up to speed on Lion.  Could you
> grab a copy of Python from python.org and use this?  It'll install under
> /Library/Frameworks/Python.framework, just add the correct bin/
> directory to the front of your $PATH and start Galaxy as normal.
> 
> Sorry for the inconvenience,
> --nate
> 
> >
> > -
> >
> > [~/galaxy-dist] myuserid 10:11 AM > ./run.sh
> > Some eggs are out of date, attempting to fetch...
> > Warning: MarkupSafe (a dependent egg of Mako) cannot be fetched
> > Warning: decorator (a dependent egg of sqlalchemy-migrate) cannot be fetched
> > Warning: simplejson (a dependent egg of WebHelpers) cannot be fetched
> > Traceback (most recent call last):
> >  File "./scripts/fetch_eggs.py", line 30, in 
> >c.resolve() # Only fetch eggs required by the config
> >  File "/Users/myuserid/galaxy-dist/lib/galaxy/eggs/__init__.py", line 345, 
> > in resolve
> >egg.resolve()
> >  File "/Users/myuserid/galaxy-dist/lib/galaxy/eggs/__init__.py", line 195, 
> > in resolve
> >return self.version_conflict( e.args[0], e.args[1] )
> >  File "/Users/myuserid/galaxy-dist/lib/galaxy/eggs/__init__.py", line 226, 
> > in version_conflict
> >r = pkg_resources.working_set.resolve( ( dist.as_requirement(), ), env, 
> > egg.fetch )
> >  File 
> > "/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/pkg_resources.py",
> >  line 565, in resolve
> >raise DistributionNotFound(req)  # XXX put more info here
> > pkg_resources.DistributionNotFound: numpy==1.6.0
> > Fetch failed.
> > [~/galaxy-dist] myuserid 10:12 AM >
> >
> >
> >
> > ___
> > Please keep all replies on the list by using "reply all"
> > in your mail client.  To manage your subscriptions to this
> > and other Galaxy lists, please use the interface at:
> >
> >   http://lists.bx.psu.edu/
> 
___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:

  http://lists.bx.psu.edu/

Re: [galaxy-dev] handling galaxy updates

[Nate Coraor]

Shantanu Pavgi wrote:
> 
> Thanks for the reply Kanwei.  Is there any announcement or news feed that 
> people can subscribe to know when main galaxy instance is updated. This will 
> help sites which want to keep up with the PSU's main galaxy instance. Do you 
> announce it on the galaxy-user list? 

This isn't currently announced, but we can start doing it via the
official Galaxy Twitter feed:

http://twitter.com/#!/galaxyproject

--nate

> 
> --
> Shantanu. 
> 
> 
> On Sep 2, 2011, at 1:57 PM, Kanwei Li wrote:
> 
> > Hi Shantanu,
> > 
> > Test usually tracks galaxy-central pretty closely, and we do update main 
> > more often than galaxy-dist (you can see the version on main at the index 
> > page). If nothing breaks on main for a while we do a galaxy-dist release so 
> > we are generally confident that it will be stable.
> > 
> > Thanks,
> > 
> > K
> > 
> > On Fri, Sep 2, 2011 at 2:22 PM, Shantanu Pavgi  wrote:
> > 
> > Hi,
> > 
> > I am curious to know which revision of galaxy code gets deployed in the 
> > main ( http://main.g2.bx.psu.edu/ ) and test ( http://test.g2.bx.psu.edu/  
> > ) instances of galaxy. I was thinking active development repository 
> > galaxy-central code gets deployed in the test galaxy instance and stable 
> > galaxy-dist code gets deployed in the main galaxy instance.  However, it 
> > seems like main galaxy instance is updated more frequently than galaxy-dist 
> > repository. Should local galaxy instances keep up with PSU's main galaxy 
> > instance revisions or wait for stable code to be released in galaxy-dist 
> > repository? We have been keeping up with galaxy-dist repository and not the 
> > galaxy-central repository updates. Are  other sites following similar 
> > update model?
> > 
> > --
> > Thanks,
> > Shantanu.
> > ___
> > Please keep all replies on the list by using "reply all"
> > in your mail client.  To manage your subscriptions to this
> > and other Galaxy lists, please use the interface at:
> > 
> >  http://lists.bx.psu.edu/
> > 
> 
> 
> ___
> Please keep all replies on the list by using "reply all"
> in your mail client.  To manage your subscriptions to this
> and other Galaxy lists, please use the interface at:
> 
>   http://lists.bx.psu.edu/
___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:

  http://lists.bx.psu.edu/

Re: [galaxy-dev] downloading pre-indexed genomes to our galaxy server

[Nate Coraor]

Nikhil Joshi wrote:
> Hi all,
> 
> Is there a repository of pre-indexed genomes for base-space/color-space,
> bwa, bam, sam, etc...?  It would be much easier than having to index them
> all ourselves :)

Hi Nik,

I've been meaning to set this up for a while now.  Stay tuned for an
announcement once it's done, we have to clean up a couple things before
it can be done.

Thanks,
--nate

> 
> - Nik.

> ___
> Please keep all replies on the list by using "reply all"
> in your mail client.  To manage your subscriptions to this
> and other Galaxy lists, please use the interface at:
> 
>   http://lists.bx.psu.edu/

___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:

  http://lists.bx.psu.edu/

Re: [galaxy-dev] (no subject)

[Nate Coraor]

Kristen Eisenberg wrote:
> [galaxy-dev] Failure to prepare job error
> 
> 
> 
> Hello,
> 
> I'm running a Python/R tool from my Amazon Galaxy instance. To use R, I
> import a package I've installed called PypeR. My tool works fine from the
> command line, but when I try to run it on my Amazon instance, it gives me
> this error:
> 
> Traceback (most recent call last):
> File "/nfs/software/galaxy/lib/galaxy/jobs/runners/local.py", line
> 61, in run_job
> job_wrapper.prepare()
> File "/nfs/software/galaxy/lib/galaxy/jobs/__init__.py", line 371, in prepare
> self.command_line = self.tool.build_command_line( param_dict )
> File "/nfs/software/galaxy/lib/galaxy/tools/__init__.py", line 1543,
> in build_command_line
> command_line = fill_template( self.command, context=param_dict )
> File "/nfs/software/galaxy/lib/galaxy/util/template.py", line 9, in
> fill_template
> return str( Template( source=template_text, searchList=[context] ) )
> File 
> "/nfs/software/galaxy/eggs/Cheetah-2.2.2-py2.6-linux-i686-ucs4.egg/Cheetah/Template.py",
> line 1004, in __str__
> return getattr(self, mainMethName)()
> File "cheetah_DynamicallyCompiledCheetahTemplate_1311891781_86_41894.py",
> line 83, in respond
> EOFError: EOF when reading a line
> 
> With the additional caption "failure to prepare job".
> I've been struggling with this for days, and I'm pretty sure I don't
> have a typo in my .xml file. How can I solve this?

Hi Kristen,

Could you post the relevant XML file to the list?  The error
unfortunately indicates a syntax error or typo.

--nate

> 
> Kristen Eisenberg
> Billige Flüge
> Marketing GmbH
> Emanuelstr. 3,
> 10317 Berlin
> Deutschland
> Telefon: +49 (33)
> 5310967
> Email:
> utebachmeier at gmail.com
> Site:
> http://flug.airego.de - Billige Flüge vergleichen

> ___
> Please keep all replies on the list by using "reply all"
> in your mail client.  To manage your subscriptions to this
> and other Galaxy lists, please use the interface at:
> 
>   http://lists.bx.psu.edu/

___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:

  http://lists.bx.psu.edu/

Re: [galaxy-dev] Path to tool designation in tool config xml

[Nate Coraor]

Timothy Wu wrote:
> Hi,
> 
> I'm trying the add tool tutorial at
> http://wiki.g2.bx.psu.edu/Admin/Tools/Add%20Tool%20Tutorial . It works, but
> how do I designate the path of the tool if it's not dropped into the same
> directory as the tool config xml? Some executable may not be in the
> directory. Surely it couldn't possiblely be the case with external tools
> like BLAST, isn't it?! But looking at the BLAST config xmls I couldn't make
> out the difference and I didn't find any documentation on how to designate
> the path of the tool. Help appreciated, thank you.

Hi Timothy,

If the command.py args... tag is
used in your tool config, the command line will be assembled as:

python /absolute/path/to/directory/containing/tool.xml/command.py args...

In this case, the interpreter (python) is expected to be found on the
$PATH.

If no interpreter is specified, i.e. binary args...
then `binary` is expected to be found on the $PATH.

Hope this helps,
--nate

> 
> Timothy

> ___
> Please keep all replies on the list by using "reply all"
> in your mail client.  To manage your subscriptions to this
> and other Galaxy lists, please use the interface at:
> 
>   http://lists.bx.psu.edu/

___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:

  http://lists.bx.psu.edu/

Re: [galaxy-dev] Quotas

[Nate Coraor]

SHAUN WEBB wrote:
> 
> Hi.
> 
> I am interested in putting quotas in to effect on our local Galaxy
> server. From the admin page I can create a quota but this only seems
> to apply to disk space. I noticed on your test site you have quotas
> for concurrent jobs also. Have I missed how to set this or is this
> functionality not available in galaxy-dist yet?

Hi Shaun,

This functionality is not as configurable as it should be, but can be
set for all users site-wide with:

user_job_limit = X

In the Galaxy config file.  I held off on adding it to the sample config
since the method to configure it is likely to change.

> 2 other issues:
> 
> 1) My account says I am using "0 bytes" in the top right corner
> although my history displays 456mb of data?

Run galaxy-dist/scripts/set_user_disk_usage.py to populate these totals.

> 2) Are quotas set by default? I was able to create quotas before I
> had updated universe_wsgi.ini to include enable_quotas = True

They exist in the interface and database  but are ignored if
enable_quotas = False.

Thanks for trying them out!  I hope to have documentation for them this
week.

--nate

> 
> 
> Shaun
> 
> -- 
> The University of Edinburgh is a charitable body, registered in
> Scotland, with registration number SC005336.
> 
> 
> ___
> Please keep all replies on the list by using "reply all"
> in your mail client.  To manage your subscriptions to this
> and other Galaxy lists, please use the interface at:
> 
>  http://lists.bx.psu.edu/
___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:

  http://lists.bx.psu.edu/

Re: [galaxy-dev] disk space and file formats

[Nate Coraor]

Peter Cock wrote:
> On Tue, Sep 6, 2011 at 3:24 PM, Nate Coraor  wrote:
> > Edward Kirton wrote:
> >> Peter wrote:
> >> > I wonder if Galaxy would benefit from a new fastqsanger-gzip (etc) 
> >> > datatype?
> >> > However this seems generally useful (not just for FASTQ) so perhaps a 
> >> > more
> >> > general mechanism would be better where tool XML files can say which file
> >> > types they accept and which of those can/must be compressed (possily not
> >> > just gzip format?).
> >>
> >> Perhaps we can flesh-out what more general solutions would look like...
> >>
> >> Imagine the fastq datatypes were left alone and instead there's a
> >> mechanism by which files which haven't been used as input for x days
> >> get compressed by a cron job.  the file server knows how to uncompress
> >> such files on the fly when needed.  For the most part, files are
> >> uncompressed during analysis and are compressed when the files exist
> >> as an archive within galaxy.
> >
> > Ideally, there'd just be a column on the dataset table indicating
> > whether the dataset is compressed or not, and then tools get a new
> > way to indicate whether they can directly read compressed inputs, or
> > whether the input needs to be decompressed first.
> >
> > --nate
> 
> Yes, that's what I was envisioning Nate.
> 
> Are there any schemes other than gzip which would make sense?
> Perhaps rather than a boolean column (compressed or not), it
> should specify the kind of compression if any (e.g. gzip).

Makes sense.

> We need something which balances compression efficiency (size)
> with decompression speed, while also being widely supported in
> libraries for maximum tool uptake.

Yes, and there's a side effect of allowing this: you may decrease
efficiency if the tools used downstream all require decompression, and
you waste a bunch of time decompressing the dataset multiple times.

--nate

> 
> Peter
> 
___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:

  http://lists.bx.psu.edu/

[galaxy-dev] bug: unsorted bam files and import into data library

[Florian Wagner]

Hi,

when a tool outputs an unsorted bam file, the indexing fails (quietly) 
and its metadata variable "bam_index" points to an inexistent file. This 
causes a nasty bug when trying to import the dataset into a data library 
and actually makes the library unusable unless you delete the broken 
entry from the - in my case Postgresql - database by hand. Are you 
working on it?


Thanks, Florian
___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:

 http://lists.bx.psu.edu/

Re: [galaxy-dev] disk space and file formats

[Peter Cock]

On Tue, Sep 6, 2011 at 5:00 PM, Nate Coraor  wrote:
> Peter Cock wrote:
>> On Tue, Sep 6, 2011 at 3:24 PM, Nate Coraor  wrote:
>> > Ideally, there'd just be a column on the dataset table indicating
>> > whether the dataset is compressed or not, and then tools get a new
>> > way to indicate whether they can directly read compressed inputs, or
>> > whether the input needs to be decompressed first.
>> >
>> > --nate
>>
>> Yes, that's what I was envisioning Nate.
>>
>> Are there any schemes other than gzip which would make sense?
>> Perhaps rather than a boolean column (compressed or not), it
>> should specify the kind of compression if any (e.g. gzip).
>
> Makes sense.
>
>> We need something which balances compression efficiency (size)
>> with decompression speed, while also being widely supported in
>> libraries for maximum tool uptake.
>
> Yes, and there's a side effect of allowing this: you may decrease
> efficiency if the tools used downstream all require decompression,
> and you waste a bunch of time decompressing the dataset multiple
> times.

While decompression wastes CPU time and makes things slower,
there is less data IO from disk (which may be network mounted)
which makes things faster. So overall, depending on the setup
and the task at hand, it could be faster.

Is it time to file an issue on bitbucket to track this potential
enhancement?

Peter
___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:

  http://lists.bx.psu.edu/

Re: [galaxy-dev] disk space and file formats

[Nate Coraor]

Peter Cock wrote:
> On Tue, Sep 6, 2011 at 5:00 PM, Nate Coraor  wrote:
> > Peter Cock wrote:
> >> On Tue, Sep 6, 2011 at 3:24 PM, Nate Coraor  wrote:
> >> > Ideally, there'd just be a column on the dataset table indicating
> >> > whether the dataset is compressed or not, and then tools get a new
> >> > way to indicate whether they can directly read compressed inputs, or
> >> > whether the input needs to be decompressed first.
> >> >
> >> > --nate
> >>
> >> Yes, that's what I was envisioning Nate.
> >>
> >> Are there any schemes other than gzip which would make sense?
> >> Perhaps rather than a boolean column (compressed or not), it
> >> should specify the kind of compression if any (e.g. gzip).
> >
> > Makes sense.
> >
> >> We need something which balances compression efficiency (size)
> >> with decompression speed, while also being widely supported in
> >> libraries for maximum tool uptake.
> >
> > Yes, and there's a side effect of allowing this: you may decrease
> > efficiency if the tools used downstream all require decompression,
> > and you waste a bunch of time decompressing the dataset multiple
> > times.
> 
> While decompression wastes CPU time and makes things slower,
> there is less data IO from disk (which may be network mounted)
> which makes things faster. So overall, depending on the setup
> and the task at hand, it could be faster.
> 
> Is it time to file an issue on bitbucket to track this potential
> enhancement?

Sure.

> 
> Peter
___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:

  http://lists.bx.psu.edu/

[galaxy-dev] Galaxy API question: workflow_execute

[Roxanne Tapia]

I was trying to use the workflow_execute.py

And I can't understand the last argument:

usage: workflow_execute.py key url workflow_id history step=src=dataset_id

So in  step=src=dataset_id
step:  I think equals the next step #in the history? 
src: ??? 
dataset_id:   This has to be the dataset id from a data library which I can get 
by using display.py to query libraries & contents of libraries. Right?  

Can anyone explain this to me? 

Roxanne Tapia
Software Developer 3
B-6, Genome Science
Bioscience Division
r...@lanl.gov
(505)665-0206





___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:

  http://lists.bx.psu.edu/

[galaxy-dev] Galaxy API question:

[Roxanne Tapia]

Hi, 

I was going through the examples in the README, and everything went along fine 
until I tried to upload files from the import directory:

usage: library_upload_from_import_dir.py key url folder_id file_type server_dir 
dbkey

I don't understand what the last two arguments 'server_dir' and 'dbkey' 
represent, or how to determine the correct values to pass along. 

I can, however, successfully upload via the import dir from the webapp. 

Any help would be appreciated, 


Roxanne Tapia
Software Developer 3
B-6, Genome Science
Bioscience Division
LANL




___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:

  http://lists.bx.psu.edu/

[galaxy-dev] Galaxy API question - histories

[Roxanne Tapia]

Is there a way to create/view histories via the api, and to associate data in 
libraries to histories? 
Then to share the histories or make them public, and build a URL to view the 
history? 

Roxanne Tapia
Software Developer 3
B-6, Genome Science
Bioscience Division
r...@lanl.gov
(505)665-0206





___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:

  http://lists.bx.psu.edu/

Re: [galaxy-dev] Galaxy API question:

[Nate Coraor]

Roxanne Tapia wrote:
> Hi, 
> 
> I was going through the examples in the README, and everything went along 
> fine until I tried to upload files from the import directory:
> 
> usage: library_upload_from_import_dir.py key url folder_id file_type 
> server_dir dbkey
> 
> I don't understand what the last two arguments 'server_dir' and 'dbkey' 
> represent, or how to determine the correct values to pass along. 
> 
> I can, however, successfully upload via the import dir from the webapp. 
> 
> Any help would be appreciated, 

Hi Roxanne,

server_dir is the name of the directory underneath your configured
'library_import_dir' in the Galaxy config file from which you would like
to import.  In the UI, these names appear in a drop-down list.

The dbkey is the "key" of the genome build that should be associated
with the data you're importing (e.g. 'hg18'), the full list of which is
also available on the UI upload page.

--nate

> 
> 
> Roxanne Tapia
> Software Developer 3
> B-6, Genome Science
> Bioscience Division
> LANL
> 
> 
> 
> 
> ___
> Please keep all replies on the list by using "reply all"
> in your mail client.  To manage your subscriptions to this
> and other Galaxy lists, please use the interface at:
> 
>   http://lists.bx.psu.edu/
___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:

  http://lists.bx.psu.edu/

Re: [galaxy-dev] Galaxy API question - histories

[Nate Coraor]

Roxanne Tapia wrote:
> Is there a way to create/view histories via the api, and to associate data in 
> libraries to histories? 
> Then to share the histories or make them public, and build a URL to view the 
> history? 

Hi Roxanne.

To the first three, yes.  This was added in the most recent stable
distribution (last week), see the example script:

history_create_history.py.

The view url for display.py is:

http://server/api/histories

The import from library example is in:

import_library_dataset_to_history.py.

Sharing/permissions are not yet available via the API.

--nate

> 
> Roxanne Tapia
> Software Developer 3
> B-6, Genome Science
> Bioscience Division
> r...@lanl.gov
> (505)665-0206
> 
> 
> 
> 
> 
> ___
> Please keep all replies on the list by using "reply all"
> in your mail client.  To manage your subscriptions to this
> and other Galaxy lists, please use the interface at:
> 
>   http://lists.bx.psu.edu/
___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:

  http://lists.bx.psu.edu/

Re: [galaxy-dev] mycoplasma genomes - suggestion

[Jennifer Jackson]

Hello Lukasz,

This is a good idea, but perhaps it would be better to share the data in 
a history where others can gain access rather than add the merged 
"contamination" genomes as a native reference genome (to avoid 
confusion). This would also give you control over the actual contents 
for updates, etc.


A direct way to do this is: create/load the contamination data in fasta 
format into a dedicated history and label appropriately (using pencil 
icon to reach the "Edit Attributes" description and info fields). Then 
use "Options -> Share or Publish" with the "Make History Accessible and 
Publish" button. You may even consider creating a "Page" to describe the 
contents and embed a link to the published history. Documenting what is 
in the fasta file and what the intended use it will make the data more 
useable for others.


Hopefully this is helpful. Others are welcome to comment about the best 
ways to share/use this type of data.


Best,

Jen
Galaxy team

On 8/29/11 8:17 AM, Lukasz Kielpinski wrote:

Hello everybody,

I would like to suggest including mycoplasma genomes (preferably all
genomes in one file) as reference genomes. It would enable fast
checking if cells used for deep-seq experiments were not contaminated.

Best,
Lukasz
___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:

   http://lists.bx.psu.edu/


--
Jennifer Jackson
http://usegalaxy.org
http://galaxyproject.org/Support
___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:

 http://lists.bx.psu.edu/

Re: [galaxy-dev] Galaxy API question: workflow_execute

[Dannon Baker]

The 'step' in question is the actual workflow step id, since ordering of steps 
in a workflow is flexible and might be changed without realizing it by moving 
steps around in the editor.  The easiest way to retrieve this identifier is to 
use the API and view the workflow in question.

Here's an example of that output, using scripts/api/display.py

$ python display.py  
http://localhost:8080/api/workflows/ebfb8f50c6abde6d
Member Information
--
url: /api/workflows/ebfb8f50c6abde6d
inputs: {'16': {'value': '', 'label': 'Input Dataset'}}
id: ebfb8f50c6abde6d
name: Foo User's Workflow

For inputs, you have a dictionary containing (in this case just one) the 
workflow steps, their labels, etc.  So, regarding the workflow_execute last 
argument, it's the step id from above, and the source is 'ldda', 'ld', or 
'hda', depending on what you're using as an input.  For inputs you're pulling 
out of a history, use 'hda'.  The other options are for using datasets directly 
from data libraries.

-Dannon

On Sep 6, 2011, at 1:24 PM, Roxanne Tapia wrote:

> I was trying to use the workflow_execute.py
> 
> And I can't understand the last argument:
> 
> usage: workflow_execute.py key url workflow_id history step=src=dataset_id
> 
> So in  step=src=dataset_id
> step:  I think equals the next step #in the history? 
> src: ??? 
> dataset_id:   This has to be the dataset id from a data library which I can 
> get by using display.py to query libraries & contents of libraries. Right?  
> 
> Can anyone explain this to me? 
> 
> Roxanne Tapia
> Software Developer 3
> B-6, Genome Science
> Bioscience Division
> r...@lanl.gov
> (505)665-0206
> 
> 
> 
> 
> 
> ___
> Please keep all replies on the list by using "reply all"
> in your mail client.  To manage your subscriptions to this
> and other Galaxy lists, please use the interface at:
> 
>  http://lists.bx.psu.edu/

___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:

  http://lists.bx.psu.edu/

Re: [galaxy-dev] Galaxy API question: workflow_execute

[Nate Coraor]

Dannon Baker wrote:
> The 'step' in question is the actual workflow step id, since ordering of 
> steps in a workflow is flexible and might be changed without realizing it by 
> moving steps around in the editor.  The easiest way to retrieve this 
> identifier is to use the API and view the workflow in question.
> 
> Here's an example of that output, using scripts/api/display.py
> 
>   $ python display.py  
> http://localhost:8080/api/workflows/ebfb8f50c6abde6d
>   Member Information
>   --
>   url: /api/workflows/ebfb8f50c6abde6d
>   inputs: {'16': {'value': '', 'label': 'Input Dataset'}}
>   id: ebfb8f50c6abde6d
>   name: Foo User's Workflow
> 
> For inputs, you have a dictionary containing (in this case just one) the 
> workflow steps, their labels, etc.  So, regarding the workflow_execute last 
> argument, it's the step id from above, and the source is 'ldda', 'ld', or 
> 'hda', depending on what you're using as an input.  For inputs you're pulling 
> out of a history, use 'hda'.  The other options are for using datasets 
> directly from data libraries.
> 
> -Dannon
> 
> On Sep 6, 2011, at 1:24 PM, Roxanne Tapia wrote:
> 
> > I was trying to use the workflow_execute.py
> > 
> > And I can't understand the last argument:
> > 
> > usage: workflow_execute.py key url workflow_id history step=src=dataset_id
> > 
> > So in  step=src=dataset_id
> > step:  I think equals the next step #in the history? 
> > src: ??? 
> > dataset_id:   This has to be the dataset id from a data library which I can 
> > get by using display.py to query libraries & contents of libraries. Right?  

In addition to what Dannon said above, there's currently a confusing
difference with respect to the dataset_id.  This is most easily pulled
from the URL of a dataset in the library or history UI, rather than the
API.  For the details on why the ID in the API won't work, see this
thread from a couple weeks ago:

http://lists.bx.psu.edu/pipermail/galaxy-dev/2011-August/006310.html

We'll have a fix for this at some point.

--nate

> > 
> > Can anyone explain this to me? 
> > 
> > Roxanne Tapia
> > Software Developer 3
> > B-6, Genome Science
> > Bioscience Division
> > r...@lanl.gov
> > (505)665-0206
> > 
> > 
> > 
> > 
> > 
> > ___
> > Please keep all replies on the list by using "reply all"
> > in your mail client.  To manage your subscriptions to this
> > and other Galaxy lists, please use the interface at:
> > 
> >  http://lists.bx.psu.edu/
> 
> ___
> Please keep all replies on the list by using "reply all"
> in your mail client.  To manage your subscriptions to this
> and other Galaxy lists, please use the interface at:
> 
>   http://lists.bx.psu.edu/
___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:

  http://lists.bx.psu.edu/

Re: [galaxy-dev] Galaxy egg fetching error? Mac OS X 10.7 (Lion)

[Whyte, Jeffrey]

Hi Nate,

Thanks for the response.  Here is the output:

[~] myuserid 01:34 PM > python ./scripts/get_platforms.py
/Library/Frameworks/Python.framework/Versions/2.7/Resources/Python.app/Contents/MacOS/Python:
 can't open file './scripts/get_platforms.py': [Errno 2] No such file or 
directory
[~] myuserid 01:34 PM >

Jeff


From: Nate Coraor [n...@bx.psu.edu]
Sent: Tuesday, September 06, 2011 10:18 AM
To: Whyte, Jeffrey
Cc: galaxy-dev@lists.bx.psu.edu
Subject: Re: [galaxy-dev] Galaxy egg fetching error? Mac OS X 10.7 (Lion)

Whyte, Jeffrey wrote:
> Thanks for the information, Nate.
>
> I installed Python as you described, and the error I get now is pasted below. 
>  I appreciate your help.
>
> Jeff
>
> [~/galaxy-dist] myuserid 03:06 PM > ./run.sh
> Some eggs are out of date, attempting to fetch...
> Warning: MarkupSafe (a dependent egg of Mako) cannot be fetched
> Warning: simplejson (a dependent egg of WebHelpers) cannot be fetched
> pysam 0.4.2 couldn't be downloaded automatically.  You can try
> building it by hand with:
>   python scripts/scramble.py -e pysam
> Fetch failed.
> [~/galaxy-dist] myuserid 03:06 PM >

Could you report back with the output of:

% python ./scripts/get_platforms.py

?

Thanks,
--nate

> 
> From: Nate Coraor [n...@bx.psu.edu]
> Sent: Friday, September 02, 2011 1:16 PM
> To: Whyte, Jeffrey
> Cc: galaxy-dev@lists.bx.psu.edu
> Subject: Re: [galaxy-dev] Galaxy egg fetching error? Mac OS X 10.7 (Lion)
>
> Whyte, Jeffrey wrote:
> > Hi,
> >
> > I've been having trouble running a local instance of Galaxy on a Mac Pro 
> > after upgrading to OS X 10.7 (Lion).  My Python version is 2.7.1 and 
> > Mercurial is 1.9.1 for MacOS X 10.7.  I don't have any MacPorts installed.
> >
> > The error I see after running the startup script is pasted at the end of 
> > this message.  Thanks in advance for any help or advice.
> > jjw
>
> Hi Jeffrey,
>
> We haven't yet gotten our dependencies up to speed on Lion.  Could you
> grab a copy of Python from python.org and use this?  It'll install under
> /Library/Frameworks/Python.framework, just add the correct bin/
> directory to the front of your $PATH and start Galaxy as normal.
>
> Sorry for the inconvenience,
> --nate
>
> >
> > -
> >
> > [~/galaxy-dist] myuserid 10:11 AM > ./run.sh
> > Some eggs are out of date, attempting to fetch...
> > Warning: MarkupSafe (a dependent egg of Mako) cannot be fetched
> > Warning: decorator (a dependent egg of sqlalchemy-migrate) cannot be fetched
> > Warning: simplejson (a dependent egg of WebHelpers) cannot be fetched
> > Traceback (most recent call last):
> >  File "./scripts/fetch_eggs.py", line 30, in 
> >c.resolve() # Only fetch eggs required by the config
> >  File "/Users/myuserid/galaxy-dist/lib/galaxy/eggs/__init__.py", line 345, 
> > in resolve
> >egg.resolve()
> >  File "/Users/myuserid/galaxy-dist/lib/galaxy/eggs/__init__.py", line 195, 
> > in resolve
> >return self.version_conflict( e.args[0], e.args[1] )
> >  File "/Users/myuserid/galaxy-dist/lib/galaxy/eggs/__init__.py", line 226, 
> > in version_conflict
> >r = pkg_resources.working_set.resolve( ( dist.as_requirement(), ), env, 
> > egg.fetch )
> >  File 
> > "/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/pkg_resources.py",
> >  line 565, in resolve
> >raise DistributionNotFound(req)  # XXX put more info here
> > pkg_resources.DistributionNotFound: numpy==1.6.0
> > Fetch failed.
> > [~/galaxy-dist] myuserid 10:12 AM >
> >
> >
> >
> > ___
> > Please keep all replies on the list by using "reply all"
> > in your mail client.  To manage your subscriptions to this
> > and other Galaxy lists, please use the interface at:
> >
> >   http://lists.bx.psu.edu/
>

___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:

  http://lists.bx.psu.edu/

Re: [galaxy-dev] get data interface for public Galaxy instance

[Daniel Blankenberg]

Hi Scott,

For information on adding Datasource tools to Galaxy, please see:
http://wiki.g2.bx.psu.edu/Admin/Internals/Data%20Sources
http://www.ncbi.nlm.nih.gov/pubmed/21531983

If your resource is using software that is already Galaxy-aware, you can often 
take an existing xml configuration (GALAXY_ROOT/tools/data_source/*.xml) and 
make a copy to modify for your needs. Please let us know if we can provide 
additional information.


Thanks for using Galaxy,

Dan


On Sep 4, 2011, at 10:29 AM, Scott Emrich wrote:

> Dear team,
> 
> There are multiple 3rd party sources to import arthropod vector data (e.g. 
> BioMart) from our NIH Bioinformatics Resource Center (BRC) VectorBase into 
> Galaxy, but some are outdated and some of our users would like a direct 
> source.  Given this increase of Galaxy in our vector biology community, what 
> is the policy for adding a new server to "Get Data" and is there information 
> on formatting our results appropriately (i.e., using a REST interface)?   Our 
> NIH BRC server is www.vectorbase.org.
> 
> I noticed our sister BRC EuPathDB has made their data available via 
> http://main.g2.bx.psu.edu, and we have similar developer resources to make 
> our server compatible; however, I couldn't find this information online 
> besides addressing this list. 
> 
> Thanks in advance, and best regards,
> 
> - S
> 
> ---
> 
> Scott Emrich, PhD
> Scientific Manager, VectorBase
> Assistant Professor, Computer Science and Engineering
> University of Notre Dame
> ___
> Please keep all replies on the list by using "reply all"
> in your mail client.  To manage your subscriptions to this
> and other Galaxy lists, please use the interface at:
> 
>  http://lists.bx.psu.edu/


___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:

  http://lists.bx.psu.edu/

Re: [galaxy-dev] TEMP variable in cluster install

[Chorny, Ilya]

Hi Nate,

I wrote the code such that if a machine had a large local scratch space then 
you could comment it out and it would go to the default setting of writing to 
/tmp or whatever is defined by SGE. You are correct that the environmental 
variables defined in the Galaxy startup script do not get passed when running 
as an actual user. Further I separated TMPDIR from new_files_path because 
new_files_path is used by Galaxy for other reasons such as uploading files and 
metadata. 

Best,

Ilya


-Original Message-
From: Nate Coraor (n...@bx.psu.edu) [mailto:n...@bx.psu.edu] 
Sent: Tuesday, September 06, 2011 7:21 AM
To: Chorny, Ilya
Cc: Assaf Gordon; Shantanu Pavgi; galaxydev psu
Subject: Re: [galaxy-dev] TEMP variable in cluster install

Chorny, Ilya wrote:
> Nate,
> 
> We ran into this issue with  /tmp not having enough space. We came up with a 
> generic solution and were wondering if you might want to add it to 
> galaxy-central. We did not want to modify SGE as it could affect other jobs 
> that use the cluster. 
> 
> We created a TMPDIR  option in universe_wsgi.ini and then modified drama.py 
> to include an "export TMPDIR=TMPDIR" to the drm_tmplate if TMPDIR is defined 
> in universe_wsgi.ini.  Do you want to pull this code into galaxy central. 
> It's like 5 lines of code. 

This is also something you can put in the Galaxy user's ~/.sge_request, 
although with your actual user code this would be more difficult.

There's actually already a temp directory option, new_file_path, this would 
just need to be added to the template.  But I don't know that it's a good idea, 
since each node may have a large local scratch space and the path to that space 
may differ (this is the case in our environment, where we actually run jobs on 
multiple clusters).

> 
>  Thanks and have a good weekend,
> 
> Ilya
> 
> BTW, any luck reviewing the run as actual user code?

I should be doing it today.

--nate

> 
> 
> -Original Message-
> From: galaxy-dev-boun...@lists.bx.psu.edu 
> [mailto:galaxy-dev-boun...@lists.bx.psu.edu] On Behalf Of Assaf Gordon
> Sent: Thursday, August 04, 2011 2:31 PM
> To: Shantanu Pavgi; galaxydev psu
> Subject: Re: [galaxy-dev] TEMP variable in cluster install
> 
> As a follow-up for SGE + TMP variables,
> 
> I've also encountered problems with settings TEMP,TMP and TMPDIR variables.
> Tried setting "~/.sge_request" and changing the Galaxy environment variables 
> before starting python - nothing worked - TMPDIR and TMP were always set to 
> "/tmp/all.q.X" by SGE .
> 
> What finally worked is changing the SGE configuration queue, and simply 
> setting the "tmpdir" variable to my desired temp directory.
> 
> Do that by running:
> $ sudo qconf -mq all.q
> 
> and changing the line:
>   tmpdir/tmp
> to
>   tmpdir/my/temporary/directory/path
> 
> 
> Problem solved :)
> No more messing around with TMP,TMPDIR variables in any Galaxy related source 
> files.
> 
> Hope this helps,
>  -gordon
> 
> 
> 
> 
> Shantanu Pavgi wrote, On 07/26/2011 05:40 PM:
> > 
> > On Jul 21, 2011, at 6:24 PM, Shantanu Pavgi wrote:
> > 
> >>
> >> We have configured galaxy to work with our SGE cluster using drmaa 
> >> job runner interface. We are using 'unified method' for this install and 
> >> both TEMP environment variable and new_file_path in universe_wsgi.ini file 
> >> have been configured correctly. However, we are seeing some errors where 
> >> local /tmp space on compute nodes is being referenced by the galaxy tools. 
> >> Specifically we saw it mentioned in error messages from following tools:
> >> * bwa_wrapper and upload tools: 'No space left on device: /tmp...' 
> >> * sam_to_bam tool: 'Error sorting alignments from /tmp/filename..'
> >>
> >> Shouldn't it be referencing TEMP environment variable or new_file_path 
> >> configuration value? Is it getting overridden by TMP or TMPDIR variables 
> >> in python wrapper code? Has anyone else experienced similar issue? 
> >>
> >>
> > 
> > Further debugging showed that there are two more temporary directory 
> > related environment variables - TMP and TMPDIR which were pointing local 
> > /tmp location on compute nodes. We tried to set these variables in our 
> > shell environment (drmaa URL uses -V to export current shell env) however 
> > SGE overwrote TMP and TMPDIR before actual job execution. The TEMP variable 
> > remained unmodified by SGE scheduler.
> > 
> > The galaxy tools seemed to be using temporary directory space pointed by 
> > TMP and TMPDIR and hence we saw local /tmp related errors mentioned in my 
> > earlier post. 
> > 
> > We have temporarily fixed this problem by hard coding TEMP, TMP and 
> > TMPDIR values in job template files - 1.
> > lib/galaxy/jobs/runners/drmaa.py 2. lib/galaxy/jobs/runners/sge.py
> > 
> > This will affect all jobs submitted through the galaxy. It will be helpful 
> > to know how it can be handled in the tool wrapper scripts. It seems like it 
> > can be handled in the tool wrapper scripts using d

Re: [galaxy-dev] get data interface for public Galaxy instance

[W. Augustine Dunn III]

I suppose what I meant to say was that since I was able to add the
metazoa.ensembl.org biomart by modifying the existing biomart xml, would
this not also work for VB's biomart instance?  Or would this not be what you
are looking for?

Actually, re-reading your OP, is it that you are asking for galaxy to add
this source to their public instances?

Gus

On Tue, Sep 6, 2011 at 11:54 AM, Daniel Blankenberg  wrote:

> Hi Scott,
>
> For information on adding Datasource tools to Galaxy, please see:
> http://wiki.g2.bx.psu.edu/Admin/Internals/Data%20Sources
> http://www.ncbi.nlm.nih.gov/pubmed/21531983
>
> If your resource is using software that is already Galaxy-aware, you can
> often take an existing xml configuration
> (GALAXY_ROOT/tools/data_source/*.xml) and make a copy to modify for your
> needs. Please let us know if we can provide additional information.
>
>
> Thanks for using Galaxy,
>
> Dan
>
>
> On Sep 4, 2011, at 10:29 AM, Scott Emrich wrote:
>
> > Dear team,
> >
> > There are multiple 3rd party sources to import arthropod vector data
> (e.g. BioMart) from our NIH Bioinformatics Resource Center (BRC) VectorBase
> into Galaxy, but some are outdated and some of our users would like a direct
> source.  Given this increase of Galaxy in our vector biology community, what
> is the policy for adding a new server to "Get Data" and is there information
> on formatting our results appropriately (i.e., using a REST interface)?
> Our NIH BRC server is www.vectorbase.org.
> >
> > I noticed our sister BRC EuPathDB has made their data available via
> http://main.g2.bx.psu.edu, and we have similar developer resources to make
> our server compatible; however, I couldn't find this information online
> besides addressing this list.
> >
> > Thanks in advance, and best regards,
> >
> > - S
> >
> > ---
> >
> > Scott Emrich, PhD
> > Scientific Manager, VectorBase
> > Assistant Professor, Computer Science and Engineering
> > University of Notre Dame
> > ___
> > Please keep all replies on the list by using "reply all"
> > in your mail client.  To manage your subscriptions to this
> > and other Galaxy lists, please use the interface at:
> >
> >  http://lists.bx.psu.edu/
>
>
> ___
> Please keep all replies on the list by using "reply all"
> in your mail client.  To manage your subscriptions to this
> and other Galaxy lists, please use the interface at:
>
>  http://lists.bx.psu.edu/
>



-- 
In science, "fact" can only mean "confirmed to such a degree that it would
be perverse to withhold provisional assent." I suppose that apples might
start to rise tomorrow, but the possibility does not merit equal time in
physics classrooms.
*-Stephen Jay Gould*
___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:

  http://lists.bx.psu.edu/

Re: [galaxy-dev] Galaxy egg fetching error? Mac OS X 10.7 (Lion)

[Whyte, Jeffrey]

Nate,

For some reason, ~/galaxy-dist wasn't in my PATH.  Here is the output when the 
script is run from the ~/galaxy-dist directory:

[~/galaxy-dist] myuserid 03:50 PM > python ./scripts/get_platforms.py
macosx-10.6-intel-ucs2
[~/galaxy-dist] myuserid 03:50 PM > 

Looks like it is set for Mac OS X 10.6 Snow Leopard instead of 10.7 Lion.  (?)

Thanks,
Jeff 


On Sep 6, 2011, at 10:18 AM, Nate Coraor wrote:

> Whyte, Jeffrey wrote:
>> Thanks for the information, Nate.
>> 
>> I installed Python as you described, and the error I get now is pasted 
>> below.  I appreciate your help.
>> 
>> Jeff
>> 
>> [~/galaxy-dist] myuserid 03:06 PM > ./run.sh
>> Some eggs are out of date, attempting to fetch...
>> Warning: MarkupSafe (a dependent egg of Mako) cannot be fetched
>> Warning: simplejson (a dependent egg of WebHelpers) cannot be fetched
>> pysam 0.4.2 couldn't be downloaded automatically.  You can try
>> building it by hand with:
>>  python scripts/scramble.py -e pysam
>> Fetch failed.
>> [~/galaxy-dist] myuserid 03:06 PM >
> 
> Could you report back with the output of:
> 
>% python ./scripts/get_platforms.py
> 
> ?
> 
> Thanks,
> --nate
> 
>> 
>> From: Nate Coraor [n...@bx.psu.edu]
>> Sent: Friday, September 02, 2011 1:16 PM
>> To: Whyte, Jeffrey
>> Cc: galaxy-dev@lists.bx.psu.edu
>> Subject: Re: [galaxy-dev] Galaxy egg fetching error? Mac OS X 10.7 (Lion)
>> 
>> Whyte, Jeffrey wrote:
>>> Hi,
>>> 
>>> I've been having trouble running a local instance of Galaxy on a Mac Pro 
>>> after upgrading to OS X 10.7 (Lion).  My Python version is 2.7.1 and 
>>> Mercurial is 1.9.1 for MacOS X 10.7.  I don't have any MacPorts installed.
>>> 
>>> The error I see after running the startup script is pasted at the end of 
>>> this message.  Thanks in advance for any help or advice.
>>> jjw
>> 
>> Hi Jeffrey,
>> 
>> We haven't yet gotten our dependencies up to speed on Lion.  Could you
>> grab a copy of Python from python.org and use this?  It'll install under
>> /Library/Frameworks/Python.framework, just add the correct bin/
>> directory to the front of your $PATH and start Galaxy as normal.
>> 
>> Sorry for the inconvenience,
>> --nate
>> 
>>> 
>>> -
>>> 
>>> [~/galaxy-dist] myuserid 10:11 AM > ./run.sh
>>> Some eggs are out of date, attempting to fetch...
>>> Warning: MarkupSafe (a dependent egg of Mako) cannot be fetched
>>> Warning: decorator (a dependent egg of sqlalchemy-migrate) cannot be fetched
>>> Warning: simplejson (a dependent egg of WebHelpers) cannot be fetched
>>> Traceback (most recent call last):
>>> File "./scripts/fetch_eggs.py", line 30, in 
>>>   c.resolve() # Only fetch eggs required by the config
>>> File "/Users/myuserid/galaxy-dist/lib/galaxy/eggs/__init__.py", line 345, 
>>> in resolve
>>>   egg.resolve()
>>> File "/Users/myuserid/galaxy-dist/lib/galaxy/eggs/__init__.py", line 195, 
>>> in resolve
>>>   return self.version_conflict( e.args[0], e.args[1] )
>>> File "/Users/myuserid/galaxy-dist/lib/galaxy/eggs/__init__.py", line 226, 
>>> in version_conflict
>>>   r = pkg_resources.working_set.resolve( ( dist.as_requirement(), ), env, 
>>> egg.fetch )
>>> File 
>>> "/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/pkg_resources.py",
>>>  line 565, in resolve
>>>   raise DistributionNotFound(req)  # XXX put more info here
>>> pkg_resources.DistributionNotFound: numpy==1.6.0
>>> Fetch failed.
>>> [~/galaxy-dist] myuserid 10:12 AM >
>>> 
>>> 
>>> 
>>> ___
>>> Please keep all replies on the list by using "reply all"
>>> in your mail client.  To manage your subscriptions to this
>>> and other Galaxy lists, please use the interface at:
>>> 
>>>  http://lists.bx.psu.edu/
>> 


___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:

  http://lists.bx.psu.edu/

Re: [galaxy-dev] Galaxy megablastdb indexer script proposal

[Jennifer Jackson]

Hello Jacob,

This would be a welcomed addition to the Tool Shed. Please see this wiki 
for instructions about how to submit:

http://galaxyproject.org/wiki/Tool%20Shed

Apologies for the late reply,

Best,

Jen
Galaxy team

On 6/23/11 5:59 PM, Jacob Israel Cervantes Luevano (LANGEBIO) wrote:

Hi Galaxy Dev,

Nice to meet you.

My name is Jacob and I am from México , Actually I'm working in LANGEBIO
(Laboratorio Nacional de Genomica para la Biodiversidad) at Bioinformatics.

I would like to contribute with galaxy dev team, I developed shell
script to set up reference genomes by megablast.

the linux shell script that I prepared uses your current prepdb.py script.

the mini pipeline flow-work:

1. read input values:
* fasta filename (nucleotide)
* blast format: blast format: F->nucleotide, T->protein
* database output name
* fasta sequence IDs format (ncbi-gbk , other)

where:

ncbi-gbk means something like ">gi|gi-number|gb|accesión|locus"
other means something like ">my own IDs"

2. if fasta sequence IDs are like ncbi format(ncbi-gk) then execute
prepdb.py python script and generate new fasta file then go step 4
example:

 >gi|gi-number|gb|accesión|locus

3. if fasta sequence IDs has diferent (other) identifiers set to galaxy
format first then execute prepdb.py python script then go step 4

 >my own identifier

4. to format database using formatdb blast utility

5. setting up Megablast Galaxy Loc File

5. delete temp files

6. done

Usage example:

query help:
./set_galaxy_blastdb.sh -help

format blast nucleotide dabatabase yeast, set galaxy loc file
./set_galaxy_blastdb.sh -fasta
/media/xlinux/genomes/galaxy/palomero/megablast/yeast_nt.fasta -p F -n
yeastdb -fastadesc ncbi-gk

when script finished we have following results:

$ls /media/xlinux/genomes/galaxy/palomero/megablast/
blastdb.loc new.yeast_nt.fasta yeastdb.nhr yeastdb.nin yeastdb.nsq
yeast_nt.fasta

How can I contribute with this shell script ?

Thanks in advance

--
Jacob
http://www.langebio.cinvestav.mx/bioinformatica/jacob

--
This message has been scanned for viruses and
dangerous content by *MailScanner* , and is
believed to be clean.


___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:

   http://lists.bx.psu.edu/


--
Jennifer Jackson
http://usegalaxy.org
http://galaxyproject.org/Support
___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:

 http://lists.bx.psu.edu/

Re: [galaxy-dev] get data interface for public Galaxy instance

[Scott Emrich]

I am asking for galaxy to add our VectorBase source to their public instance, 
as VectorBase sourced-data (either biomart instance) may be out-of-date.  The 
UCSC data is even older.  As EuPath has their latest release data available 
from the public interface, we were interested in doing the same.

This delay should be at most two months, though, from talking to EBI so we'll 
advocate VectorBase users to use the biomart xml/public interface and you 
should be fine Gus.   I'll look at the XML spec tomorrow just in case.

- S

On Sep 6, 2011, at 3:40 PM, W. Augustine Dunn III wrote:

I suppose what I meant to say was that since I was able to add the 
metazoa.ensembl.org biomart by modifying the 
existing biomart xml, would this not also work for VB's biomart instance?  Or 
would this not be what you are looking for?

Actually, re-reading your OP, is it that you are asking for galaxy to add this 
source to their public instances?

Gus

On Tue, Sep 6, 2011 at 11:54 AM, Daniel Blankenberg 
mailto:d...@bx.psu.edu>> wrote:
Hi Scott,

For information on adding Datasource tools to Galaxy, please see:
http://wiki.g2.bx.psu.edu/Admin/Internals/Data%20Sources
http://www.ncbi.nlm.nih.gov/pubmed/21531983

If your resource is using software that is already Galaxy-aware, you can often 
take an existing xml configuration (GALAXY_ROOT/tools/data_source/*.xml) and 
make a copy to modify for your needs. Please let us know if we can provide 
additional information.


Thanks for using Galaxy,

Dan


On Sep 4, 2011, at 10:29 AM, Scott Emrich wrote:

> Dear team,
>
> There are multiple 3rd party sources to import arthropod vector data (e.g. 
> BioMart) from our NIH Bioinformatics Resource Center (BRC) VectorBase into 
> Galaxy, but some are outdated and some of our users would like a direct 
> source.  Given this increase of Galaxy in our vector biology community, what 
> is the policy for adding a new server to "Get Data" and is there information 
> on formatting our results appropriately (i.e., using a REST interface)?   Our 
> NIH BRC server is www.vectorbase.org.
>
> I noticed our sister BRC EuPathDB has made their data available via 
> http://main.g2.bx.psu.edu, and we have similar 
> developer resources to make our server compatible; however, I couldn't find 
> this information online besides addressing this list.
>
> Thanks in advance, and best regards,
>
> - S
>
> ---
>
> Scott Emrich, PhD
> Scientific Manager, VectorBase
> Assistant Professor, Computer Science and Engineering
> University of Notre Dame
> ___
> Please keep all replies on the list by using "reply all"
> in your mail client.  To manage your subscriptions to this
> and other Galaxy lists, please use the interface at:
>
>  http://lists.bx.psu.edu/


___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:

 http://lists.bx.psu.edu/



--
In science, "fact" can only mean "confirmed to such a degree that it would be 
perverse to withhold provisional assent." I suppose that apples might start to 
rise tomorrow, but the possibility does not merit equal time in physics 
classrooms.
-Stephen Jay Gould


___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:

  http://lists.bx.psu.edu/

Re: [galaxy-dev] [galaxy-bugs] Error message when trying to dowload Manhattan plot using the web version

[Jennifer Jackson]

Hello Alison,

I wanted to share with you the bitbucket ticket for the improvements to 
composite datatypes. As Ross described, this would be part of a larger 
project. The ability to save composite data using the floppydisc icon 
would be one example an impact of the proposed change.


https://bitbucket.org/galaxy/galaxy-central/issue/654/improvements-for-composite-datatypes

Thanks again for the feedback and use case example,

Best,

Jen
Galaxy team

On 7/19/11 8:44 PM, Ross wrote:

Hi, Alison

I've made some fixes to the Manhattan/QQ tool (galaxy-central revision
#dacf97a6c663) so it now handles your data - thanks again for
discovering these edge cases - the revised code is now available and
working on http://test.g2.bx.psu.edu/ where your data produces plots.
In particular X in your data, and for good measure Y and Mt are now
translated into 23,24 and 25 respectively so the code can now cope
with those inputs and they should appear in linear order in the
Manhattan x axis

The floppydisk link (download) problem remains to be fixed as it's
more complex. We only update the main server every few weeks so the
changes I just made will take a while to appear there. Meanwhile,
please try the revised tool by making yourself an account and then
uploading your data to http://test.g2.bx.psu.edu/ - please let me know
how you get on?

Of course, the (?FDR adjusted) p values in your data are a challenge -
typically, the Manhattan and QQ plots are run with unadjusted p values
which are expected to be distributed uniformly 0-1 under the null...
FYI, when I use the grouping tool on your data, I see that chromosomes
20,21 and 22 are absent from the file - is this expected? Group with a
count gives:

1   22910
10  9990
11  12764
12  13413
13  13068
14  11319
15  11175
16  8483
17  12017
18  10073
19  6927
2   18332
3   16173
4   15401
5   15259
6   15487
7   15217
8   12485
9   14560
X   3951

Thanks again for helping us improve Galaxy...

On Mon, Jul 18, 2011 at 10:52 PM, Alison Harrill  wrote:

The first time I uploaded the file, I loaded in the -logP values
instead. After that, I changed the file to raw p values and reuploaded.
Do you think it would help to upload the file again, but change the
filename? I just wonder if that original file didn't get cleared out.

Alison

-Original Message-
From: Ross [mailto:ross.laza...@gmail.com]
Sent: Monday, July 18, 2011 8:51 AM
To: Alison Harrill
Subject: Re: [galaxy-bugs] Error message when trying to dowload
Manhattan plot using the web version

ok - thanks. These are not raw pvalues and your data is definitely
causing the Manhattan plot to fail.

This will take me a little time to figure out - I'll get back to you
in a day or two - thanks again for reporting this problem

On Mon, Jul 18, 2011 at 10:16 PM, Alison Harrill
  wrote:

Good morning,

Sorry for taking a while to get back to you (I was out of town for the
weekend). I set permissions to share the dataset with you. Please let

me

know if you see an obvious workaround.

Thank you,
Alison


Alison Harrill, Ph.D.
Research Investigator
Institute for Drug Safety Sciences
Hamner Institutes for Health Sciences
phone: (919) 226-3142
fax: (919) 226-3150



-Original Message-
From: Ross [mailto:ross.laza...@gmail.com]
Sent: Sunday, July 17, 2011 7:25 AM
To: Alison Harrill
Subject: Re: [galaxy-bugs] Error message when trying to dowload
Manhattan plot using the web version

Can you please share a history with me (use this email address) so I
can take a look please?

I'm talking about links on the results page where you (should!) see
some graphs - when you click the history item display ('eye') icon -
not the history item save ('floppy disk') icon

On Sun, Jul 17, 2011 at 9:15 PM, Alison Harrill



wrote:

Hi, Ross,

Thanks so much for kindly following up with me. I tried the

workaround

you suggested, but it appears to not be working for me.

Right-clicking

prompts me to save a file called display.htm and when I open the

file,

it just reads "Unable to remove temporary library download archive

and

directory".

Best wishes,
Alison

-Original Message-
From: Ross [mailto:ross.laza...@gmail.com]
Sent: Sunday, July 17, 2011 4:09 AM
To: Alison Harrill
Subject: Re: [galaxy-bugs] Error message when trying to dowload
Manhattan plot using the web version

Hi, Alison - I wanted to be sure you are able to get your images from
the tool - the proper fix is going to take a while so I hope those
links are working for you?

cheers...


On Sat, Jul 16, 2011 at 9:39 AM, Ross  wrote:

Hi, Alison,

Thanks for reporting this error - I just confirmed that you have

found

a bug - the download function is currently not working correctly for
composite datatypes in the history.

Fortunately, the images themselves have download links under each

one

and can be saved - eg on a PC or linux workstation, while hovering
over the '(Click to download imag

Re: [galaxy-dev] Path to tool designation in tool config xml

[Timothy Wu]

Thanks, this helps. I was running with an interpreter and I didn't realize I
had to had the full path in there. I was expecting that $PATH which contains
the directory the script was located in to take care of it. Thanks you very
much for the clarification.

Timothy
___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:

  http://lists.bx.psu.edu/