[galaxy-dev] August 2014 Galaxy Newsletter

[Dave Clements]

Hello all,

The August 2014 Galaxy Update Newsletter
 is out. Here's a
preview of what's inside:

   -

   GCC2015 will be held in Norwich, United Kingdom, 6-8 July 2015
   

   -

   GCC2014 was a tremendous success
   .
   Watch the talks, review the slides, posters, and BoF writeups.
   -

   ISMB and BOSC 2014 slides, posters and videos
   

too.
   -

   53 new papers
   
   -

   One new public server (IM-PET)
   
   -

   Who's hiring?
    It
   turns out that lots of places are.

Happy August,

Dave Clements and the Galaxy Team

-- 
http://galaxyproject.org/
http://getgalaxy.org/
http://usegalaxy.org/
https://wiki.galaxyproject.org/
___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  http://lists.bx.psu.edu/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/

Re: [galaxy-dev] new_file_path questions

[John Chilton]

On Fri, Aug 1, 2014 at 4:16 AM, Geert Vandeweyer
 wrote:
> Hi,
>
> I'm running into issues with our galaxy installation regarding the
> new_file_path setting:
>
> 1/ clean up :
> 
> Can I safely delete all contents in the tmp folder using a cronjob, or
> manually when galaxy is (not) running. I have gathered almost 700Gb and >72K
> files on data there apparently. I suppose they are remnants of crashed jobs

Mostly yes. The only caveat versus what you said is I wouldn't assume
you can just delete files because Galaxy isn't running (unless you are
using the local job manager) - so I would just delete files by age
(anything older than 2 weeks or a month or something like that).

>
> 2/ Should it be accessible by all nodes
> ===
> Since I have that much data there, I suppose it does. However, it would make
> sense to use a seperate (local /tmp/ ?) folder for toolshed actions such as
> installations. As an example, I tried to install the package_perl from the
> test toolshed. On our production server, hosting the new_file_path on a NFS
> share with the 700GB / 72K files, this process ran for over 24 hours, then
> crashed the main process with the error below. On my development box
> (laptop, no network drives, single galaxy process), the package installed in
> under five minutes.

Traditionally it has needed to be shared across the compute nodes and
Galaxy because "extra" collected/discovered files were populated there
- but I think all the dev team tools now write those files to the
working directory instead. So I cannot think of a reason this needs to
be shared - I assume it can be something like /tmp but I could
certainly be wrong (anyone else have a more informed opinion?).

I am not sure what to say about the PERL error... I hadn't even
occurred to me that the install framework used this directory. Hope
the rest of my response was somewhat helpful though.

Hope this helps,
-John

>
>
> Best,
>
> Geert
>
> ==
>
>
> tool_shed.galaxy_install.install_manager ERROR 2014-08-01 05:10:20,656 Error
> installing tool dependency perl version 5.18.1.
> Traceback (most recent call last):
>   File
> "/galaxy/galaxy-dist/lib/tool_shed/galaxy_install/install_manager.py", line
> 110, in install_and_build_package_via_fabric
> tool_dependency = self.install_and_build_package( app,
> tool_shed_repository, tool_dependency, actions_dict )
>   File
> "/galaxy/galaxy-dist/lib/tool_shed/galaxy_install/install_manager.py", line
> 94, in install_and_build_package
> initial_download=False )
>   File
> "/galaxy/galaxy-dist/lib/tool_shed/galaxy_install/tool_dependencies/recipe/recipe_manager.py",
> line 32, in execute_step
> initial_download=initial_download )
>   File
> "/galaxy/galaxy-dist/lib/tool_shed/galaxy_install/tool_dependencies/recipe/step_handler.py",
> line 510, in execute_step
> return_output=False )
>   File
> "/galaxy/galaxy-dist/lib/tool_shed/galaxy_install/tool_dependencies/recipe/install_environment.py",
> line 148, in handle_command
> context.flush()
>   File
> "/galaxy/galaxy-dist/eggs/SQLAlchemy-0.7.9-py2.6-linux-x86_64-ucs4.egg/sqlalchemy/orm/scoping.py",
> line 114, in do
> return getattr(self.registry(), name)(*args, **kwargs)
>   File
> "/galaxy/galaxy-dist/eggs/SQLAlchemy-0.7.9-py2.6-linux-x86_64-ucs4.egg/sqlalchemy/orm/session.py",
> line 1718, in flush
> self._flush(objects)
>   File
> "/galaxy/galaxy-dist/eggs/SQLAlchemy-0.7.9-py2.6-linux-x86_64-ucs4.egg/sqlalchemy/orm/session.py",
> line 1807, in _flush
> transaction.rollback(_capture_exception=True)
>   File
> "/galaxy/galaxy-dist/eggs/SQLAlchemy-0.7.9-py2.6-linux-x86_64-ucs4.egg/sqlalchemy/orm/session.py",
> line 386, in rollback
> transaction._rollback_impl()
>   File
> "/galaxy/galaxy-dist/eggs/SQLAlchemy-0.7.9-py2.6-linux-x86_64-ucs4.egg/sqlalchemy/orm/session.py",
> line 414, in _rollback_impl
> t[1].rollback()
>   File
> "/galaxy/galaxy-dist/eggs/SQLAlchemy-0.7.9-py2.6-linux-x86_64-ucs4.egg/sqlalchemy/engine/base.py",
> line 2034, in rollback
> self._do_rollback()
>   File
> "/galaxy/galaxy-dist/eggs/SQLAlchemy-0.7.9-py2.6-linux-x86_64-ucs4.egg/sqlalchemy/engine/base.py",
> line 2071, in _do_rollback
> self.connection._rollback_impl()
>   File
> "/galaxy/galaxy-dist/eggs/SQLAlchemy-0.7.9-py2.6-linux-x86_64-ucs4.egg/sqlalchemy/engine/base.py",
> line 1265, in _rollback_impl
> self._handle_dbapi_exception(e, None, None, None, None)
>   File
> "/galaxy/galaxy-dist/eggs/SQLAlchemy-0.7.9-py2.6-linux-x86_64-ucs4.egg/sqlalchemy/engine/base.py",
> line 1262, in _rollback_impl
> self.engine.dialect.do_rollback(self.connection)
>   File
> "/galaxy/galaxy-dist/eggs/SQLAlchemy-0.7.9-py2.6-linux-x86_64-ucs4.egg/sqlalchemy/dialects/mysql/base.py",
> line 1910, in do_rollback
> connection.rollback()
> OperationalError: (OperationalError) (2006, 'MySQL server has gone away')
> None None
> tool_shed.util.common_install_util ERROR 2014-08-01 05:10:20,743 Error
> installing

Re: [galaxy-dev] TestToolShed failure, Exception: History in error state.

[John Chilton]

On Thu, Jul 31, 2014 at 12:29 PM, Peter Cock  wrote:
> On Thu, Jul 31, 2014 at 5:21 PM, bjoern.gruen...@googlemail.com
>  wrote:
>> Hi Peter,
>>
>>
>> 2014-07-31 10:57 GMT+02:00 Peter Cock :
>>
>>> Hi Dave,
>>>
>>> You are right that on closer inspection I've mixed tool_dependencies.xml
>>> and repository_dependencies.xml *again*. Evidentially my mental model
>>> does not match Greg's here:
>>>
>>> (*) I need to define a tool installation recipe for something not in the
>>> Tool Shed --> write an install script called tool_dependencies.xml
>>>
>>> (*) I need to depend on a Python package by pointing at another repository
>>> in the Tool Shed --> repository_dependencies.xml
>>
>> I might be wrong, but I think that also goes to tool_dependencies.xml
>
> Correct, e.g.
> https://github.com/peterjc/pico_galaxy/tree/master/tools/seq_select_by_id
>
> Thanks!
>
>>>
>>> (*) I need to depend on a datatype package by pointing at another
>>> repository in the Tool Shed --> repository_dependencies.xml
>>>
>>> (*) I need to depend on a binary package by pointing at another repository
>>> in the Tool Shed --> repository_dependencies.xml ? No. You need
>>> tool_dependencies.xml for this too
>>
>> As far as I understood, everything that is referenced in the tool.xml under
>> the requirement section, needs to be in a tool_dependencies.xml file. Any
>> other dependency are from the repository (data_types, data_manager,
>> workflows ...).
>>
>> Ciao,
>> Bjoern
>
> Sure, there is a logic here - but its a definition which I seem to still
> struggle with :(
>
>>> But that aside, the test framework error here is completely unhelpful.
>>>
>>> Why is there no error message about missing a dependency?
>>> Was there an error from running my tool which was not shown?
>>>
>>> Thanks,
>>>
>>> Peter
>
> I'd still like to get a more explicit error from the test suite than
> "History in error state" though ;)

Yes - that is problematic.

Dave B - can we set GALAXY_TEST_VERBOSE_ERRORS to "True" in the
environment of the install and test framework? I disabled it by
default because it causes to sqlite to fallover rather frequently -
but if install and test framework is backed by a postgres database
this is probably not a problem.

-John

>
> Regards,
>
> Peter
> ___
> Please keep all replies on the list by using "reply all"
> in your mail client.  To manage your subscriptions to this
> and other Galaxy lists, please use the interface at:
>   http://lists.bx.psu.edu/
>
> To search Galaxy mailing lists use the unified search at:
>   http://galaxyproject.org/search/mailinglists/
___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  http://lists.bx.psu.edu/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/

[galaxy-dev] bx version conflict causes failure for some tools (e.g. subtract interval tool)

[Langhorst, Brad]

see:
https://biostar.usegalaxy.org/p/8095/

I can work around the problem by adding a stdio block to these tools, how 
should the underlying problem be solved?


Brad


--
Bradley W. Langhorst, Ph.D.
Applications and Product Development Scientist

___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  http://lists.bx.psu.edu/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/

[galaxy-dev] Error while installing tools from main toolshed: "HTTPerror : HTTP error 307 : temporary redirect"

[Perley, Danielle]

Hi,


 I'm trying to install tools from the main toolshed, but I keep getting the 
same error regardless of the tool that I'm installing


HTTPError: HTTP Error 307: Temporary Redirect
URL: 
http://localhost:8080/admin_toolshed/prepare_for_install?tool_shed_url=https://toolshed.g2.bx.psu.edu/&repository_ids=2a5f494c5fbb2c90&changeset_revisions=bc9269529e88
Module weberror.evalexception.middleware:364 in respond  view
>>  app_iter = self.application(environ, detect_start_response)
Module paste.recursive:84 in __call__  view
>>  return self.application(environ, start_response)
Module paste.httpexceptions:633 in __call__  view
>>  return self.application(environ, start_response)
Module galaxy.web.framework.base:132 in __call__  view
>>  return self.handle_request( environ, start_response )
Module galaxy.web.framework.base:190 in handle_request  view
>>  body = method( trans, **kwargs )
Module galaxy.web.framework:377 in decorator  view
>>  return func( self, trans, *args, **kwargs )
Module galaxy.webapps.galaxy.controllers.admin_toolshed:1116 in 
prepare_for_install  view
>>  updating=updating )
Module tool_shed.util.common_install_util:110 in 
get_dependencies_for_repository  view
>>  all_repo_info_dict = get_required_repo_info_dicts( trans, tool_shed_url, 
>> util.listify( repo_info_dict ) )
Module tool_shed.util.common_install_util:451 in get_required_repo_info_dicts   
   view
>>  response = urllib2.urlopen( request ).read()
Module urllib2:127 in urlopen  view
>>  return _opener.open(url, data, timeout)
Module urllib2:410 in open  view
>>  response = meth(req, response)
Module urllib2:523 in http_response  view
>>  'http', request, response, code, msg, hdrs)
Module urllib2:442 in error  view
>>  result = self._call_chain(*args)
Module urllib2:382 in _call_chain  view
>>  result = func(*args)
Module urllib2:608 in http_error_302  view
>>  new = self.redirect_request(req, fp, code, msg, headers, newurl)
Module urllib2:569 in redirect_request  view
>>  raise HTTPError(req.get_full_url(), code, msg, headers, fp)
HTTPError: HTTP Error 307: Temporary Redirect



I've just installed Galaxy locally for the first time on a Mac OS 10.9.3. The 
only changes I've made to the universe_wsg.ini file are:

setting host=0.0.0.0 (The error also occurred with host=127.0.0.1?)

specifying a full path for Tool_depenendency_dir

giving myself administrator privileges with admin_users=myemail


any ideas?


Danielle

___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  http://lists.bx.psu.edu/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/

Re: [galaxy-dev] Toolshed tool_dependency.xml git archive configure issue

[Dooley, Damion]

That was it! Yay!  I'll add a Trello card to have this info gem added to the 
docs.

Thanks,

Damion

Hsiao lab, BC Public Health Microbiology & Reference Laboratory, BC Centre for 
Disease Control
655 West 12th Avenue, Vancouver, British Columbia, V5Z 4R4 Canada

From: Aaron Petkau [aaron.pet...@gmail.com]
Sent: Friday, August 01, 2014 12:26 PM
To: Dooley, Damion
Cc: galaxy-dev@lists.bx.psu.edu
Subject: Re: [galaxy-dev] Toolshed tool_dependency.xml git archive configure 
issue

Hmmm... you know what.  I tested this out and it looks like Galaxy expects the 
git repo to be named the same as the package name in your tool dependencies 
file ffp-phylogeny.  If you change the git command to:

git clone https://github.com/apetkau/ffp-3.19-custom.git ffp-phylogeny

I think it should work.

Aaron
___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  http://lists.bx.psu.edu/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/

Re: [galaxy-dev] Toolshed tool_dependency.xml git archive configure issue

[Aaron Petkau]

Hmmm... you know what.  I tested this out and it looks like Galaxy expects
the git repo to be named the same as the package name in your tool
dependencies file ffp-phylogeny.  If you change the git command to:

git clone https://github.com/apetkau/ffp-3.19-custom.git *ffp-phylogeny*

I think it should work.

Aaron


On Fri, Aug 1, 2014 at 2:09 PM, Dooley, Damion 
wrote:

> A little more info:
>
> ls -a
>
> #
> ls -a
> STDOUT
> .
> ..
> #
>
> #
> ls -a
> STDERR
>
> so I guess the .git archive is being unpackaged somewhere else!  After
> lunch I'll return to explore this mysterious territory.
>
> d.
___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  http://lists.bx.psu.edu/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/

Re: [galaxy-dev] Toolshed tool_dependency.xml git archive configure issue

[Dooley, Damion]

A little more info:

ls -a 

#
ls -a
STDOUT
.
..
#

#
ls -a
STDERR

so I guess the .git archive is being unpackaged somewhere else!  After lunch 
I'll return to explore this mysterious territory.

d. 
___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  http://lists.bx.psu.edu/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/

Re: [galaxy-dev] Toolshed tool_dependency.xml git archive configure issue

[Dooley, Damion]

Sadly,

 ./configure --disable-gui 
--prefix=$INSTALL_DIR

yeilds install error:

 /bin/sh: ./configure: No such file or directory

But file must be there since previous git step is carried out ok:
...
#
git reset --hard 1819482db93bf0025c2588b5264cc0ec8c61029c
STDOUT
HEAD is now at 1819482 Added gitignore

#
git reset --hard 1819482db93bf0025c2588b5264cc0ec8c61029c
STDERR

#
./configure --disable-gui 
--prefix=/usr/local/galaxy/shared/tool_dependency/ffp-phylogeny/0.3.19_1819482db93bf0025c2588b5264cc0ec8c61029c/ddooley/ffp_phylogeny/4e213eee0fef
STDOUT

#
./configure --disable-gui 
--prefix=/usr/local/galaxy/shared/tool_dependency/ffp-phylogeny/0.3.19_1819482db93bf0025c2588b5264cc0ec8c61029c/ddooley/ffp_phylogeny/4e213eee0fef
STDERR
/bin/sh: ./configure: No such file or directory
#

argh!

d.

From: Aaron Petkau [aaron.pet...@gmail.com]
Sent: Friday, August 01, 2014 11:47 AM
To: Dooley, Damion
Cc: galaxy-dev@lists.bx.psu.edu
Subject: Re: [galaxy-dev] Toolshed tool_dependency.xml git archive configure 
issue

Hey Damion,

You should be able to do ./configure --disable-gui ... to get it to work (that 
is, make sure the full path to the configure command is given).  At least I 
think that's probably what's going on.

Aaron


___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  http://lists.bx.psu.edu/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/

Re: [galaxy-dev] Toolshed tool_dependency.xml git archive configure issue

[Aaron Petkau]

Hey Damion,

You should be able to do *./*configure --disable-gui ... to get it to work
(that is, make sure the full path to the configure command is given).  At
least I think that's probably what's going on.

Aaron


On Fri, Aug 1, 2014 at 12:32 PM, Dooley, Damion 
wrote:

> Hi folks,
>
> Having a difficult time with coding the "configure" step in this new
> tool_dependency.xml file, namely that an error "/bin/sh: configure: command
> not found" results when I try to install my beta development tool from a
> local test toolshed (which is downloading and installing other tools just
> fine).
>
> 
> git clone
> https://github.com/myuser/mygitstuff.git .
> git reset --hard
> 1819482db93bf0025c2588b5264cc0ec8c61029c
> configure --disable-gui
> --prefix=$INSTALL_DIR
> 
> 
> bin
>
> $INSTALL_DIR/bin
> 
> 
>  action="prepend_to">$INSTALL_DIR/bin
> 
> 
>
> Any hints about why this isn't being found or what the /bin/sh needs to
> get it going?
> When I do a git reset --hard, the "configure" file is definitely there,
> with executable flag set.  Any other permission oddities?
>
> I've also tried  , which yeilds same
> error.  (Skipping that line and doing 
> at least finds and runs "make", though it errors out for lack of input
> data).
>
> Regards,
>
> Damion
>
> Hsiao lab, BC Public Health Microbiology & Reference Laboratory, BC Centre
> for Disease Control
> 655 West 12th Avenue, Vancouver, British Columbia, V5Z 4R4 Canada
>
> ___
> Please keep all replies on the list by using "reply all"
> in your mail client.  To manage your subscriptions to this
> and other Galaxy lists, please use the interface at:
>   http://lists.bx.psu.edu/
>
> To search Galaxy mailing lists use the unified search at:
>   http://galaxyproject.org/search/mailinglists/
>
___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  http://lists.bx.psu.edu/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/

[galaxy-dev] Toolshed tool_dependency.xml git archive configure issue

[Dooley, Damion]

Hi folks,

Having a difficult time with coding the "configure" step in this new 
tool_dependency.xml file, namely that an error "/bin/sh: configure: command not 
found" results when I try to install my beta development tool from a local test 
toolshed (which is downloading and installing other tools just fine).  


git clone 
https://github.com/myuser/mygitstuff.git .
git reset --hard 
1819482db93bf0025c2588b5264cc0ec8c61029c 
configure --disable-gui 
--prefix=$INSTALL_DIR 


bin

$INSTALL_DIR/bin


$INSTALL_DIR/bin



Any hints about why this isn't being found or what the /bin/sh needs to get it 
going?
When I do a git reset --hard, the "configure" file is definitely there, with 
executable flag set.  Any other permission oddities?

I've also tried  , which yeilds same error.  
(Skipping that line and doing  at least 
finds and runs "make", though it errors out for lack of input data).

Regards,

Damion

Hsiao lab, BC Public Health Microbiology & Reference Laboratory, BC Centre for 
Disease Control
655 West 12th Avenue, Vancouver, British Columbia, V5Z 4R4 Canada

___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  http://lists.bx.psu.edu/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/

Re: [galaxy-dev] galaxy install nginx

[Langhorst, Brad]

The order is not very important…
however I would install
nginx and postgres at the same time, then galaxy.

brad
--
Bradley W. Langhorst, Ph.D.
Applications and Product Development Scientist

On Aug 1, 2014, at 10:10 AM, 
ngsf...@hygenomics.com wrote:


dear sir:
   nginx  postgress  galaxy   which one first install ?

ngsf...@hygenomics.com
___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
 http://lists.bx.psu.edu/

To search Galaxy mailing lists use the unified search at:
 http://galaxyproject.org/search/mailinglists/

___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  http://lists.bx.psu.edu/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/

[galaxy-dev] Announcement: Galaxy IPython Integration - interactive programming in Galaxy

[Björn Grüning]

Hi all,

we proudly present the first release of the Galaxy IPython project.

Galaxy IPython is a visualization plugin which should enable Galaxy 
users with coding skills to easily process their data in the most 
flexible way. With this plugin, it is possible to analyse and 
post-process data without downloading datasets or entire histories. One 
of our aims was to make Galaxy more attractive and accessible to 
bioinformaticians and programmers, and we hope that this project will 
build some bridges.


Screencast: http://www.youtube.com/watch?v=jQDyTuYnn1k

Disclaimer: Even though the Ipython notebooks can be stored and reused, 
this plugin will break the Galaxy philosophy of reproducibility, I feel 
personally bad about that, but I also think it is a great opportunity to 
get more bioinformaticians into Galaxy, and to get Galaxy used more 
often as a teaching resource. By being able to teach not only about 
workflows but also about data analysis tasks often necessary with 
Bioinformatics, Galaxy will be significantly more useful in teaching 
environments.
Keep in mind to write a nice Tool Shed Tool if you catch yourself using 
IPython in Galaxy to often for the same task.


A few features we have up and running:
Use IPython directly in the main window or in the Scratchbook
Completely encapsulated IPython environment with matplotlib, biopython, 
pandas and friends already installed.
IPython runs completely self-contained within a docker container, 
separate from your Galaxy data
Easy access to datasets from your current history via pre-defined 
IPython functions
Manipulate and plot data as you like and export your new files back into 
your Galaxy history
Save IPython Notebooks across analysis sessions in your Galaxy history 
with the click of a button.
View saved IPython Notebooks directly in HTML format, or re-open them to 
continue your analysis.

Self-closing and self-cleaning IPython docker container
Notebooks are secure, only accessible to the intended user


Please follow the installation instruction on our project page under:
https://github.com/bgruening/galaxy-ipython

The underlying IPython Notebook (+Galaxy sugar) is stored under:
https://github.com/bgruening/docker-ipython-notebook
https://registry.hub.docker.com/u/bgruening/docker-ipython-notebook/
You can also install a ipynb datatype:
https://github.com/bgruening/galaxytools/tree/master/datatypes/json
https://testtoolshed.g2.bx.psu.edu/view/iuc/datatyp_ipynb


Comments welcome!
Happy research!
Eric & Björn
___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
 http://lists.bx.psu.edu/

To search Galaxy mailing lists use the unified search at:
 http://galaxyproject.org/search/mailinglists/

[galaxy-dev] galaxy install nginx

[ngsf...@hygenomics.com]

dear sir:
   nginx  postgress  galaxy   which one first install ? 


ngsf...@hygenomics.com
___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  http://lists.bx.psu.edu/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/

Re: [galaxy-dev] Packaging tools with language specific dependencies

[Renato Alves]

Hi BJörn,

We have run into some problems with the perl environment.
This might also affect other tools that rely on a shebang with absolute
paths to the interpreter. Details below.

During the installation of the perl package (we tried package_perl_5_18
from main and testing toolshed), the cpanm script gets installed but
when executed it is invalid/not found.

The issue seems to be the length of the shebang in the script which is
limited to 80 characters. In cpanm it's:

!#/galaxy_dist/tool_dependencies/perl/5.18.1/bgruening/package_perl_5_18/e89824189ec6/bin/perl

That is 92 characters long. When launching the script it gets truncated
at "...perl_5_18/e898241", causing the command to fail with a "bad
interpreter" error.

I was able to workaround this problem by editing the script manually and
replacing the long shebang by: #!/usr/bin/env perl


As it stands it seems relying on the shebang to use the correct
interpreter is a problem. Repositories with long (character wise) names,
versions or owners will be more likely to suffer from this. So will
installations that are not close to the system root.

Renato

Quoting Björn Grüning on 30-07-2014 12:04:
> Hi Renato,
> 
> Am 30.07.2014 um 12:21 schrieb Renato Alves:
>> Hi everyone,
>>
>> Is there any standard or commonly used way to package tools that have
>> language specific dependencies.
>>
>> I know that with Python libraries one can use setup_virtualenv and with
>> Java jars the JAVA_JAR_LIB strategy is used.
>> Is there anything equivalent for R, Perl and Ruby libraries?
> 
> please have a look at:
> https://github.com/bgruening/galaxytools/tree/master/test_repositories
> 
> If you have any questions I'm happy to help you!
> Bjoern
> 
>> Thanks
>> Renato
>>
>>
>>
>>
>>
>> ___
>> Please keep all replies on the list by using "reply all"
>> in your mail client.  To manage your subscriptions to this
>> and other Galaxy lists, please use the interface at:
>>http://lists.bx.psu.edu/
>>
>> To search Galaxy mailing lists use the unified search at:
>>http://galaxyproject.org/search/mailinglists/
>>



signature.asc
Description: OpenPGP digital signature
___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  http://lists.bx.psu.edu/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/

[galaxy-dev] new_file_path questions

[Geert Vandeweyer]

Hi,

I'm running into issues with our galaxy installation regarding the 
new_file_path setting:


1/ clean up :

Can I safely delete all contents in the tmp folder using a cronjob, or 
manually when galaxy is (not) running. I have gathered almost 700Gb and 
>72K files on data there apparently. I suppose they are remnants of 
crashed jobs


2/ Should it be accessible by all nodes
===
Since I have that much data there, I suppose it does. However, it would 
make sense to use a seperate (local /tmp/ ?) folder for toolshed actions 
such as installations. As an example, I tried to install the 
package_perl from the test toolshed. On our production server, hosting 
the new_file_path on a NFS share with the 700GB / 72K files, this 
process ran for over 24 hours, then crashed the main process with the 
error below. On my development box (laptop, no network drives, single 
galaxy process), the package installed in under five minutes.



Best,

Geert

==


tool_shed.galaxy_install.install_manager ERROR 2014-08-01 05:10:20,656 
Error installing tool dependency perl version 5.18.1.

Traceback (most recent call last):
  File 
"/galaxy/galaxy-dist/lib/tool_shed/galaxy_install/install_manager.py", 
line 110, in install_and_build_package_via_fabric
tool_dependency = self.install_and_build_package( app, 
tool_shed_repository, tool_dependency, actions_dict )
  File 
"/galaxy/galaxy-dist/lib/tool_shed/galaxy_install/install_manager.py", 
line 94, in install_and_build_package

initial_download=False )
  File 
"/galaxy/galaxy-dist/lib/tool_shed/galaxy_install/tool_dependencies/recipe/recipe_manager.py", 
line 32, in execute_step

initial_download=initial_download )
  File 
"/galaxy/galaxy-dist/lib/tool_shed/galaxy_install/tool_dependencies/recipe/step_handler.py", 
line 510, in execute_step

return_output=False )
  File 
"/galaxy/galaxy-dist/lib/tool_shed/galaxy_install/tool_dependencies/recipe/install_environment.py", 
line 148, in handle_command

context.flush()
  File 
"/galaxy/galaxy-dist/eggs/SQLAlchemy-0.7.9-py2.6-linux-x86_64-ucs4.egg/sqlalchemy/orm/scoping.py", 
line 114, in do

return getattr(self.registry(), name)(*args, **kwargs)
  File 
"/galaxy/galaxy-dist/eggs/SQLAlchemy-0.7.9-py2.6-linux-x86_64-ucs4.egg/sqlalchemy/orm/session.py", 
line 1718, in flush

self._flush(objects)
  File 
"/galaxy/galaxy-dist/eggs/SQLAlchemy-0.7.9-py2.6-linux-x86_64-ucs4.egg/sqlalchemy/orm/session.py", 
line 1807, in _flush

transaction.rollback(_capture_exception=True)
  File 
"/galaxy/galaxy-dist/eggs/SQLAlchemy-0.7.9-py2.6-linux-x86_64-ucs4.egg/sqlalchemy/orm/session.py", 
line 386, in rollback

transaction._rollback_impl()
  File 
"/galaxy/galaxy-dist/eggs/SQLAlchemy-0.7.9-py2.6-linux-x86_64-ucs4.egg/sqlalchemy/orm/session.py", 
line 414, in _rollback_impl

t[1].rollback()
  File 
"/galaxy/galaxy-dist/eggs/SQLAlchemy-0.7.9-py2.6-linux-x86_64-ucs4.egg/sqlalchemy/engine/base.py", 
line 2034, in rollback

self._do_rollback()
  File 
"/galaxy/galaxy-dist/eggs/SQLAlchemy-0.7.9-py2.6-linux-x86_64-ucs4.egg/sqlalchemy/engine/base.py", 
line 2071, in _do_rollback

self.connection._rollback_impl()
  File 
"/galaxy/galaxy-dist/eggs/SQLAlchemy-0.7.9-py2.6-linux-x86_64-ucs4.egg/sqlalchemy/engine/base.py", 
line 1265, in _rollback_impl

self._handle_dbapi_exception(e, None, None, None, None)
  File 
"/galaxy/galaxy-dist/eggs/SQLAlchemy-0.7.9-py2.6-linux-x86_64-ucs4.egg/sqlalchemy/engine/base.py", 
line 1262, in _rollback_impl

self.engine.dialect.do_rollback(self.connection)
  File 
"/galaxy/galaxy-dist/eggs/SQLAlchemy-0.7.9-py2.6-linux-x86_64-ucs4.egg/sqlalchemy/dialects/mysql/base.py", 
line 1910, in do_rollback

connection.rollback()
OperationalError: (OperationalError) (2006, 'MySQL server has gone 
away') None None
tool_shed.util.common_install_util ERROR 2014-08-01 05:10:20,743 Error 
installing tool dependency perl version 5.18.1: Can't reconnect until 
invalid transaction is rolled back (original cause: InvalidRequestError: 
Can't reconnect until invalid transaction is rolled back) 'UPDATE 
tool_dependency SET update_time=%s, status=%s, error_message=%s WHERE 
tool_dependency.id = %s' [{'status': 'Error', 'error_message': '  File 
"/galaxy/galaxy-dist/lib/tool_shed/galaxy_install/install_manager.py", 
line 110, in install_and_build_package_via_fabric\n tool_dependency = 
self.install_and_build_package( app, tool_shed_repository, 
tool_dependency, actions_dict )\n  File 
"/galaxy/galaxy-dist/lib/tool_shed/galaxy_install/install_manager.py", 
line 94, in install_and_build_package\ninitial_download=False )\n  
File 
"/galaxy/galaxy-dist/lib/tool_shed/galaxy_install/tool_dependencies/recipe/recipe_manager.py", 
line 32, in execute_step\ninitial_download=initial_download )\n File 
"/galaxy/galaxy-dist/lib/tool_shed/galaxy_install/tool_dependencies/recipe/step_handler.py", 
line 510, in execute_step\nreturn_output=False )\n  File 
"/galax