On Thu, Aug 8, 2013 at 1:31 PM, Nate Coraor <n...@bx.psu.edu> wrote:
> On Aug 5, 2013, at 6:23 PM, Rodolfo Aramayo wrote:
>
>> Hi,
>>
>> I have a 80 core/1TB RAM Dell server.
>> In order improve Galaxy performance I modified the universe_wsgi.ini
>> file so as to improve the Scaling and Load Balancing
>> as follows:
>>
>> # Use a threadpool for the web server instead of creating a thread for each
>> # request.
>> use_threadpool = True
>>
>> # Number of threads in the web server thread pool.
>> threadpool_workers = 200
>>
>> ################################################################################
>> [server:web0]
>> use = egg:Paste#http
>> port = 8090
>> host = 127.0.0.1
>> use_threadpool = true
>> threadpool_workers = 10
>
> ... snip ...
>
>> [server:handler18]
>> use = egg:Paste#http
>> port = 8318
>> host = 127.0.0.1
>> use_threadpool = true
>> threadpool_workers = 5
>>
>> # [server:handler19]
>> # use = egg:Paste#http
>> # port = 8319
>> # host = 127.0.0.1
>> # use_threadpool = true
>> # threadpool_workers = 5
>>
>> # [server:handler20]
>> # use = egg:Paste#http
>> # port = 8320
>> # host = 127.0.0.1
>> # use_threadpool = true
>> # threadpool_workers = 5
>>
>> ################################################################################
>> job_manager = manager
>> job_handlers =
>> handler0,handler1,,handler2,handler3,handler4,handler5,handler6,handler7,handler8,handler9,handler10,handler11,handler12,handler13,handler14,handler15,handler16,handler17,handler18
>
> Hi Rodolfo,
>
> The job_manager and job_handlers options go under the [server:main] section,
> it looks like they are probably underneath the last [server:...] section
> right now.
>
> Please note that all of these features should now be configured via the job
> configuration file (job_conf.xml), and I've just updated the documentation
> you referred to accordingly.
>
> --nate
Dear Nate,
Thanks
Just to be sure I understand this correctly
I now have in my 'universe' file the following:
#####
# ---- HTTP Server
----------------------------------------------------------
# Configuration of the internal HTTP server.
[server:main]
# job_manager = manager <I ASSUME THIS IS CORRECT because I get
errors when it is not commented
################################################################################
[server:manager]
use = egg:Paste#http
port = 8200
host = 127.0.0.1
use_threadpool = true
threadpool_workers = 5
################################################################################
[server:handler0]
use = egg:Paste#http
port = 8300
host = 127.0.0.1
use_threadpool = true
threadpool_workers = 5
[server:handler1]
use = egg:Paste#http
port = 8301
host = 127.0.0.1
use_threadpool = true
threadpool_workers = 5
...snip...
[server:handler19]
use = egg:Paste#http
port = 8319
host = 127.0.0.1
use_threadpool = true
threadpool_workers = 5
# [server:handler20]
# use = egg:Paste#http
# port = 8320
# host = 127.0.0.1
# use_threadpool = true
# threadpool_workers = 5
################################################################################
# The internal HTTP server to use. Currently only Paste is provided.
This
# option is required.
use = egg:Paste#http
# The port on which to listen.
#port = 8080
# The address on which to listen. By default, only listen to
localhost (Galaxy
# will not be accessible over the network). Use '0.0.0.0' to listen
on all
# available network interfaces.
#host = 127.0.0.1
host = 0.0.0.0
# Use a threadpool for the web server instead of creating a thread for
each
# request.
use_threadpool = True
# Number of threads in the web server thread pool.
threadpool_workers = 200
################################################################################
[server:web0]
use = egg:Paste#http
port = 8400
host = 127.0.0.1
use_threadpool = true
threadpool_workers = 10
[server:web1]
use = egg:Paste#http
port = 8401
host = 127.0.0.1
use_threadpool = true
threadpool_workers = 10
...snip...
[server:web11]
use = egg:Paste#http
port = 8411
host = 127.0.0.1
use_threadpool = true
threadpool_workers = 10
[server:web12]
use = egg:Paste#http
port = 8412
host = 127.0.0.1
use_threadpool = true
threadpool_workers = 10
################################################################################
# ---- Filters --------------------------------------------------------------
# Filters sit between Galaxy and the HTTP server.
...
#####
And in my job_conf.xml:
<handler id="handler0" tags="handlers"/>
<handler id="handler1" tags="handlers"/>
...snip...
<handler id="handler18" tags="handlers"/>
<handler id="handler19" tags="handlers"/>
#####
Is this correct?
Also, I guess the part I do not understand is what is the magic
formula between the number of threadpool_workers as configured here:
# Use a threadpool for the web server instead of creating a thread for
each
# request.
use_threadpool = True
# Number of threads in the web server thread pool.
threadpool_workers = 200
and those configured on the web servers and on the server handlers?
Thanks
--Rodolfo
___________________________________________________________
Please keep all replies on the list by using "reply all"
in your mail client. To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
http://lists.bx.psu.edu/
To search Galaxy mailing lists use the unified search at:
http://galaxyproject.org/search/mailinglists/
