Re: [galaxy-dev] Multiple web servers and handlers Galaxy failed

[Makis Ladoukakis]

Sorry for bumping my own question but I didn't get any replies. Has anyone 
succesfully installed a production Galaxy instance? 

I could really use some advice.

Thank you,
Makis

From: makis4e...@hotmail.com
To: galaxy-...@lists.bx.psu.edu; galaxy-dev@lists.galaxyproject.org
Date: Tue, 1 Dec 2015 18:03:44 +0200
Subject: [galaxy-dev] Multiple web servers and handlers Galaxy failed




Hello everyone,

I am trying to setup a Galaxy instance for a multi-user production environment 
and I tried to follow the instructions here:

https://wiki.galaxyproject.org/Admin/Config/Performance/Scaling
https://wiki.galaxyproject.org/Admin/Config/Performance/ProductionServer
https://wiki.galaxyproject.org/Admin/Config/ApacheProxy

but when I tried to start my galaxy instance (GALAXY_RUN_ALL=1 sh run.sh 
--daemon) I got the following error in my browser:Bad Gateway
The proxy server received an invalid
response from an upstream server.


I don't get any errors from the paster.log files so I am guessing the error is 
somewhere in my apache configuration although I can't see where it could be.


Has anyone encountered this problem before?

Kind regards,
Makis


P.S. My apache configuration is as follows:

I added that to my apache httpd.conf file:

#for galaxy
#RewriteRule ^/galaxy(.*) http://localhost:8091$1 [P]
RewriteRule ^/galaxy$ /galaxy/ [R]
RewriteRule ^/galaxy/static/style/(.*) 
/home/galaxy/galaxy/static/june_2007_style/blue/$1 [L]
RewriteRule ^/galaxy/static/scripts/(.*) 
/home/galaxy/galaxy/static/scripts/packed/$1 [L]
RewriteRule ^/galaxy/static/(.*) /home/galaxy/galaxy/static/$1 [L]
RewriteRule ^/galaxy/favicon.ico /home/galaxy/galaxy/static/favicon.ico [L]
RewriteRule ^/galaxy/robots.txt /home/galaxy/galaxy/static/robots.txt [L]

LoadModule uwsgi_module /etc/httpd/modules/mod_uwsgi.so


Sethandler uwsgi-handler
uWSGISocket 127.0.0.1:4001
uWSGImaxVars 512


I am also attaching my job_conf.xml and galaxy.ini file.

  

___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  https://lists.galaxyproject.org/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/ 
  ___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  https://lists.galaxyproject.org/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/

[galaxy-dev] unexpanded macros

[Langhorst, Brad]

Hi:
I just installed hisat2 and the data manager from the toolshed ….

Some macros seem not to be expanded when I load the tool in the UI (even across 
galaxy restarts)
e.g. I see @FTYPE@ instead of fastqsanger

the data manager produces this error:

Fatal error: Exit code 1 ()
Traceback (most recent call last):
  File 
"/mnt/galaxy/data/galaxy/galaxy-toolshed-tools/toolshed.g2.bx.psu.edu/repos/iuc/data_manager_hisat2_index_builder/98a60a4cfb9a/data_manager_hisat2_index_builder/data_manager/hisat2_index_builder.py",
 line 85, in 
main()
  File 
"/mnt/galaxy/data/galaxy/galaxy-toolshed-tools/toolshed.g2.bx.psu.edu/repos/iuc/data_manager_hisat2_index_builder/98a60a4cfb9a/data_manager_hisat2_index_builder/data_manager/hisat2_index_builder.py",
 line 79, in main
build_hisat_index( data_manager_dict, options, params, sequence_id, 
sequence_name )
  File 
"/mnt/galaxy/data/galaxy/galaxy-toolshed-tools/toolshed.g2.bx.psu.edu/repos/iuc/data_manager_hisat2_index_builder/98a60a4cfb9a/data_manager_hisat2_index_builder/data_manager/hisat2_index_builder.py",
 line 41, in build_hisat_index
proc = subprocess.Popen( args=args, shell=False, cwd=target_directory )
  File "/mnt/galaxy/data/galaxy/sw/lib/python2.7/subprocess.py", line 710, in 
__init__
errread, errwrite)
  File "/mnt/galaxy/data/galaxy/sw/lib/python2.7/subprocess.py", line 1327, in 
_execute_child
raise child_exception
OSError: [Errno 13] Permission denied


that seems to be a result of a missing execute bit for the binaries in

tool-deps/hisat/2.0/iuc/package_hisat_2_0/c65f00072e57/bin



Should I file issues, or did my installation go wrong?

Any hints to get this running?


Brad


--
Brad Langhorst, Ph.D.
Development Scientist
New England Biolabs

___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  https://lists.galaxyproject.org/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/

Re: [galaxy-dev] Publish tool with Python dependencies to the toolshed

[Dooley, Damion]

Related to this thread, my report calc tool needs a few python packages,
but I think they are generic enough to be in python 2.6, 2.7 or 3.x , so
can I specify the packages without specifying the python version? (I'm
looking at the 
https://github.com/galaxyproject/tools-iuc/blob/master/packages/package_pyt
hon_2_7_bcbiogff_0_6_2/tool_dependencies.xml file). Or do I need to be
doing something like: pip install
...  ?  I'm able to upload this to our local toolshed, and it
seems to install ok from there, but I see in test server galaxy log
"galaxy.tools.deps WARNING 2015-12-03 22:23:02,513 Failed to resolve
dependency on 'report_calc', ignoring", though tool still runs (probably
because the python packages were already installed manually previously.)

Much obliged,

Damion


Tool definition file has:


report_calc



tool_dependencies.xml :








https://pypi.python.org/packages/source/d/dateutils/dateutils
-0.6.6.tar.gz#md5=2ba7fcac03635f1f1cad0d94d785001b

https://pypi.python.org/packages/source/p/pyparsing/pyparsing
-2.0.6.tar.gz#md5=a2d85979e33a6600148c6383d3d8de67



 








___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  https://lists.galaxyproject.org/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/

Re: [galaxy-dev] 3 questions

[Olivier CLAUDE]

Hello Peter,

Thanks again for all your explanations.

 

Hello again,

 

1/

I managed to find WolfPsort on github.

 

You mean https://github.com/fmaguire/WoLFPSort ? I can see why Finlay Maguire 
did that although I'm not 100% sure that is within the licence.


In other news, it looks like someone else has bought the domain 
http://wolfpsort.org/ and is holding it to random (for sale, offers over 
$1190). Oh dear. :(

 

When i saw that the domain was for sale, i downloaded the one from github, 
thinking that it would become really hard to find it.


 

I followed the readme file and it work with the command line but when I tried 
to run it on galaxy with the one included in the package “tmhmm and signal” 
from peterjc, it gave me an error:

 

“can’t locate fastafmt/GetOptWarnHandler.pm”

 

I put in my .bashrc the path of the file. I tried to put it directly in the 
/bin directory but it didn’t change anything.

Any idea anyone?

 

Have you followed the INSTALL file instructions? Have you been able to run 
WolfPsort at the command line?

Note it only has precompiled binaries under bin/binByPlatform for i386 and 
sparc, however we could use the 32bit binaries on our 64bit Linux machine.

 

According to my old comment inside the Python wrapper script for Galaxy, I had 
trouble running the tool from outside its home directory and so used a simple 
(second) wrapper script to change directory before running the real binary. See:

https://github.com/peterjc/pico_galaxy/blob/master/tools/protein_analysis/wolf_psort.py#L43

 

 

I followed the INSTALL file. After your message I tried the second python 
script, pointing to the directory where the file are, but It did not changed 
the error message. 


 

 

 

2/in the python script I can see:

 

Num_thread = thread_count(sys.argv[2], default=4)

 

Does it means that it will use at the maximum 4 threads? Can I assume it will 
use 4 core in parallel? 

 

 

 

This means my Python wrapper script will default to 4 threads if not specified 
via the command line.

When called via Galaxy, the XML wrapper will use the $GALAXY_SLOTS environment 
variable (if set). See the job runner settings taken from job_conf.xml for 
details.

This means unless you've setup something special in Galaxy for the number of 
slots/threads to use, the tool will default to four threads. This means it will 
break up the input file into chunks and run four copies of the single threaded 
tool runWolfPsortSummary at once.


 

3/I use blast + from the devteam is there any possibility to use more than 1 
core? If yes where? 

 

 

Thanks !

 


Again, the BLAST+ wrappers will use the $GALAXY_SLOTS environment variable (if 
set), although here they default to using calling BLAST+ with 8 threads. 

 

Peter

___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  https://lists.galaxyproject.org/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/

Re: [galaxy-dev] Publish tool with Python dependencies to the toolshed

[Ulf Schaefer]

Hi again all

So I uploaded a tool with a single dependency to python2.7.10 to the 
testtoolshed 
(https://testtoolshed.g2.bx.psu.edu/view/ulfschaefer/filter_vcf/d0afc9c121a5).

Then I try to install it back on my test Galaxy instance. There are some 
error messages when trying to install the dependencies of 
package_python_2_7_10 (e.g. package readline_6_3 or package_gdbm_1_11) 
These seem to be resolved when I go into the admin interface and 
uninstall and reinstall these one by one.

However, when I try to reinstall the package_python_2_7_10 itself after 
uninstalling it, I get this:

Error - : coercing to Unicode: need string 
or buffer, NoneType found
URL: 
http://testgalaxy/admin_toolshed/manage_tool_dependencies?repository_id=d071e794759ab192&operation=install
File 
'/home/galaxy/galaxy-dist/lib/galaxy/web/framework/middleware/error.py', 
line 151 in __call__
   app_iter = self.application(environ, sr_checker)
File 
'/home/galaxy/galaxy-dist/eggs/Paste-1.7.5.1-py2.6.egg/paste/recursive.py', 
line 84 in __call__
   return self.application(environ, start_response)
File 
'/home/galaxy/galaxy-dist/eggs/Paste-1.7.5.1-py2.6.egg/paste/httpexceptions.py',
 
line 633 in __call__
   return self.application(environ, start_response)
File '/home/galaxy/galaxy-dist/lib/galaxy/web/framework/base.py', line 
133 in __call__
   return self.handle_request( environ, start_response )
File '/home/galaxy/galaxy-dist/lib/galaxy/web/framework/base.py', line 
191 in handle_request
   body = method( trans, **kwargs )
File '/home/galaxy/galaxy-dist/lib/galaxy/web/framework/decorators.py', 
line 92 in decorator
   return func( self, trans, *args, **kwargs )
File 
'/home/galaxy/galaxy-dist/lib/galaxy/webapps/galaxy/controllers/admin_toolshed.py',
 
line 891 in manage_tool_dependencies
   status=status )
File '/home/galaxy/galaxy-dist/lib/galaxy/web/framework/decorators.py', 
line 92 in decorator
   return func( self, trans, *args, **kwargs )
File 
'/home/galaxy/galaxy-dist/lib/galaxy/webapps/galaxy/controllers/admin_toolshed.py',
 
line 494 in initiate_tool_dependency_installation
   tool_shed_repository.repo_path( trans.app ) )
File '/home/galaxy/galaxy-dist/lib/tool_shed/util/hg_util.py', line 138 
in get_config_from_disk
   for root, dirs, files in os.walk( relative_install_dir ):
File '/usr/lib64/python2.6/os.py', line 276 in walk
   names = listdir(top)
TypeError: coercing to Unicode: need string or buffer, NoneType found

What have I done wrong so that relative_install_dir is not set. Wasn't 
really a problem when I installed the other dependencies.

Any help would be appreciated. I tried switching if off and on again, 
but unfortunately it did not help.

Thanks
Ulf

On 01/12/15 16:07, Martin Čech wrote:
> Hi Ulf,
>
> you most probably want your tool to depend on package_python_2_7_10 from
> the iuc (it will ensure your tool has access to 2.7.10 no matter what
> python the Galaxy executing your tool is using)
>
> The package is being developed here
> https://github.com/galaxyproject/tools-iuc/tree/master/packages/package_python_2_7_10
>  and
> is available at the MTS here
> https://toolshed.g2.bx.psu.edu/view/iuc/package_python_2_7_10/73631e818d3d
>
> You will need to add to your wrapper's xml something like:
>
> 
> python
> 
>
> And you have to include tool_dependencies.xml file which will look
> something like:
>
> 
> 
> 
>  prior_installation_required="True" />
> 
> 
>
> Please let me know if you have more questions.
>
> M.
>
> On Tue, Dec 1, 2015 at 10:54 AM Ulf Schaefer  > wrote:
>
> Dear all
>
> I have a Python script that I would like to publish to the tool shed. It
> has a couple dependencies, all Python modules (argparse, PyVCF, PyYAML).
> It also needs a Python version 2.7<=X<3, which is not available per
> default everywhere.
>
> Firstly, how do I specify these dependencies in the tool
>  tag?
>
> And secondly, once I have a correct set of tool definition files with
> tests and test data, how do I best get it onto the tool shed?
>
> I had a look through the wiki and the planemo documentation.
> Unfortunately how to handle these dependencies did not become
> immediately obvious (to me).
>
> Thanks for any pointers and sorry if this has been answered some place I
> did not look.
>
> Cheers
> Ulf
>
> --
> Ulf Schaefer, PhD
> Bioinformatics Scientist
> Bioinformatics Unit - Infectious Disease Informatics
> National Infection Service
> Public Health England
> 61 Colindale Ave, London NW9 5EQ
> ulf.schae...@phe.gov.uk 
> http://www.gov.uk/phe
> Protecting and improving the nation’s health
>
> **
> The information contained in the EMail and any attachments is
> confidential and intended solely and for the attention and use of
> the named addressee(s). It may not be di

Re: [galaxy-dev] gcc error during pixman-0.32.4 installation on Mac OS X 10.11

[Hans Rudolph]

Hi,

not sure whether that helps, but I tried to compile r-3-2-1 outside galaxy on 
the same machine:

1.straight ./configure yields error: "cannot compile a simple Fortran program."
To remedie this, I installed gfortran (=gf) 5.2 for OS X (10.11) from 
https://gcc.gnu.org/wiki/GFortranBinaries 


and with gf installed, ./configure now completes, but make fails with "dyld: 
Symbol not found: _libiconv_open"

2. I then installed libiconv-1.4 as suggested in this tip for R-on-OS X: 
http://blog.qiubio.com:8080/archives/3897 

and put /usr/local/lib into $PATH
now make finishes …


I made mistakes running ./configure and make/install
first omitting the epxort LD_LIBRARY_PATH= command … and I don’t have proper 
records ;-)

Not sure how to clean this up - but I wanted to let you know my findings with 
respect to compiling r-3-2-1

Cheers,

Hans



___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  https://lists.galaxyproject.org/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/

Re: [galaxy-dev] gcc error during pixman-0.32.4 installation on Mac OS X 10.11

[Marius van den Beek]

I opened a new PR here https://github.com/galaxyproject/tools-iuc/pull/459.
This adds the last remaining LD_INCLUDE_PATH's and as a side effect should
also help people with cairo problems.
Note that, at the core, Hans' problem stem from an outdated version of
fontconfig on the main toolshed, that is not setting FONTCONFIG_LIB_PATH
and FONTCONFIG_INCLUDE_PATH,
so updating fontconfig would suffice to get R compiling.
But let's accept the PR and move away from relying on arbitrarily named ENV
vars that are subject to change ...

On 3 December 2015 at 09:44, Marius van den Beek 
wrote:

> I could reproduce this, and in fact removing all the export statements
> would solve the problem.
> Those shouldn't be necessary in the first place since we consequently
> modify LD_INCLUDE_PATH and LD_LIBRARY_PATH.
> I have no proof yet, but I think the underlying problem is that OS Xs GCC
> compiler doesn't tolerate empty include or library PATHs. Probably one of
> the variables in the export statement is missing because it was renamed or
> removed. Yet, it would be much less maintenance work to rely on modified
> LD_INCLUDE_PATH and LD_LIBRARY_PATH and use the autoconf action.
> What do you think about this, Bjoern?
>
> Marius
>
> On 3 December 2015 at 09:21, Marius van den Beek 
> wrote:
>
>> Hi Hans,
>>
>> I've seen this error ("
>> configure: error: C compiler cannot create executables”) quite a few
>> times on OS X, and contrary to the message, is usually not related to the
>> installed compiler (all your other packages compiled just fine). I'll check
>> this on my macs, to see if we didn't introduce a regression. My feeling is
>> that one of the include PATHS hasn't been set correctly. Could you post the
>> full INSTALLATION.log ?
>>
>> Marius
>>
>> On 3 December 2015 at 00:49, Hans Rudolph  wrote:
>>
>>> Hi Björn,
>>>
>>> thanks very much for your efforts. Unfortunately, installation of the
>>> new deseq2-1-8-2 (uic, 2015-12-02) still fails for me -
>>>
>>> apparently due to the included package_r_3_2_1 which fails with "
>>> configure: error: C compiler cannot create executables”
>>> and the R error status stops deseq2 installation with error "
>>> /bin/sh: line 7: Rscript: command not found”.
>>>
>>> But all the other dependencies in the deseq2-1-8-2 package:
>>>
>>> atlas-3-10
>>> cairo-1-12-14
>>> fontconfig-2-11-1
>>> freetype-2-5-2
>>> libpng-1-6-7
>>> libxml2-2-9-1
>>> ncurses-6-0
>>> pixman-0.32-4
>>> pixman-0-32-6
>>> readline-6-3
>>>
>>> show the green label “installed” as installation status.
>>>
>>> Best regards,
>>> Hans
>>>
>>>
>>>
>>>
>>>
>>> ___
>>> Please keep all replies on the list by using "reply all"
>>> in your mail client.  To manage your subscriptions to this
>>> and other Galaxy lists, please use the interface at:
>>>   https://lists.galaxyproject.org/
>>>
>>> To search Galaxy mailing lists use the unified search at:
>>>   http://galaxyproject.org/search/mailinglists/
>>>
>>
>>
>
___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  https://lists.galaxyproject.org/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/

Re: [galaxy-dev] gcc error during pixman-0.32.4 installation on Mac OS X 10.11

[Marius van den Beek]

I could reproduce this, and in fact removing all the export statements
would solve the problem.
Those shouldn't be necessary in the first place since we consequently
modify LD_INCLUDE_PATH and LD_LIBRARY_PATH.
I have no proof yet, but I think the underlying problem is that OS Xs GCC
compiler doesn't tolerate empty include or library PATHs. Probably one of
the variables in the export statement is missing because it was renamed or
removed. Yet, it would be much less maintenance work to rely on modified
LD_INCLUDE_PATH and LD_LIBRARY_PATH and use the autoconf action.
What do you think about this, Bjoern?

Marius

On 3 December 2015 at 09:21, Marius van den Beek 
wrote:

> Hi Hans,
>
> I've seen this error ("
> configure: error: C compiler cannot create executables”) quite a few
> times on OS X, and contrary to the message, is usually not related to the
> installed compiler (all your other packages compiled just fine). I'll check
> this on my macs, to see if we didn't introduce a regression. My feeling is
> that one of the include PATHS hasn't been set correctly. Could you post the
> full INSTALLATION.log ?
>
> Marius
>
> On 3 December 2015 at 00:49, Hans Rudolph  wrote:
>
>> Hi Björn,
>>
>> thanks very much for your efforts. Unfortunately, installation of the new
>> deseq2-1-8-2 (uic, 2015-12-02) still fails for me -
>>
>> apparently due to the included package_r_3_2_1 which fails with "
>> configure: error: C compiler cannot create executables”
>> and the R error status stops deseq2 installation with error "
>> /bin/sh: line 7: Rscript: command not found”.
>>
>> But all the other dependencies in the deseq2-1-8-2 package:
>>
>> atlas-3-10
>> cairo-1-12-14
>> fontconfig-2-11-1
>> freetype-2-5-2
>> libpng-1-6-7
>> libxml2-2-9-1
>> ncurses-6-0
>> pixman-0.32-4
>> pixman-0-32-6
>> readline-6-3
>>
>> show the green label “installed” as installation status.
>>
>> Best regards,
>> Hans
>>
>>
>>
>>
>>
>> ___
>> Please keep all replies on the list by using "reply all"
>> in your mail client.  To manage your subscriptions to this
>> and other Galaxy lists, please use the interface at:
>>   https://lists.galaxyproject.org/
>>
>> To search Galaxy mailing lists use the unified search at:
>>   http://galaxyproject.org/search/mailinglists/
>>
>
>
___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  https://lists.galaxyproject.org/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/

Re: [galaxy-dev] gcc error during pixman-0.32.4 installation on Mac OS X 10.11

[Marius van den Beek]

Hi Hans,

I've seen this error ("
configure: error: C compiler cannot create executables”) quite a few times
on OS X, and contrary to the message, is usually not related to the
installed compiler (all your other packages compiled just fine). I'll check
this on my macs, to see if we didn't introduce a regression. My feeling is
that one of the include PATHS hasn't been set correctly. Could you post the
full INSTALLATION.log ?

Marius

On 3 December 2015 at 00:49, Hans Rudolph  wrote:

> Hi Björn,
>
> thanks very much for your efforts. Unfortunately, installation of the new
> deseq2-1-8-2 (uic, 2015-12-02) still fails for me -
>
> apparently due to the included package_r_3_2_1 which fails with "
> configure: error: C compiler cannot create executables”
> and the R error status stops deseq2 installation with error "
> /bin/sh: line 7: Rscript: command not found”.
>
> But all the other dependencies in the deseq2-1-8-2 package:
>
> atlas-3-10
> cairo-1-12-14
> fontconfig-2-11-1
> freetype-2-5-2
> libpng-1-6-7
> libxml2-2-9-1
> ncurses-6-0
> pixman-0.32-4
> pixman-0-32-6
> readline-6-3
>
> show the green label “installed” as installation status.
>
> Best regards,
> Hans
>
>
>
>
>
> ___
> Please keep all replies on the list by using "reply all"
> in your mail client.  To manage your subscriptions to this
> and other Galaxy lists, please use the interface at:
>   https://lists.galaxyproject.org/
>
> To search Galaxy mailing lists use the unified search at:
>   http://galaxyproject.org/search/mailinglists/
>
___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  https://lists.galaxyproject.org/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/