On Tue, 11 Oct 2011, Gregory Favre wrote: > At the very beginning I used the default values (5 wsgi processes, 100 > MySQL connections, 100 MaxKeepAliveRequests). I tried to increase the > number of concurrent wsgi processes to 80 (it was really slow before > that), but then I ran into a "Too many connections error" returned by > MySQL.
I assume you probably don't run citation indexing, so your WSGI worker processes should be rather slim and not very memory-hungry. (Check via `ps aux | grep wsgi'.) If they are indeed slim, then you should be definitely running more than 5 WSGI worker processes on the box, say 30-40 or even more. This is no problem as far as you increase your DB server parameters related to `max_connections' accordingly, and as far as there is still enough free RAM for the front-end Apache processes (that serve CSS and images and stuff, and that `load-balance' incoming requests for the back-end WSGI processes). Concerning the too-many-connections troubles that you observed with 80 WSGI processes even though the DB connection limit was set to 100, there are CLI processes such as bibsched tasks and bibsched monitor that run queries and that are to be taken into account, and perhaps you may have other apps connecting to the same DB server (Django submission system?) that also consume connections. So you should lower 80 accordingly, say to 30-50 or whatever your concrete conditions allow. As for MySQL tuning, see my old notes at: <https://twiki.cern.ch/twiki/bin/view/CDS/InvenioTuning#1_Tuning_MySQL> as well as an example DB configuration tuned for CDS and INSPIRE use cases that is presented at: <http://invenio-software.org/wiki/Installation/InvenioOnSLC5#A.1ConfigureMySQL> Briefly, you should check how big the sum of your *.MYI files is and then increase your `key_buffer' accordingly (we use 4 GB), while still leaving enough space for caching *.MYD files by the regular GNU/Linux filesystem cache. (This is essential with MyISAM tables that Invenio uses; so key_buffer should not be set too high but should remain below one third of RAM size or thereabouts.) After tuning `key_buffer' you should also increase `open_files_limit' (we use 4096) and `table_cache' (we use 1024) since Invenio uses lots of tables. Finally, you can tweak `query_cache_size' (we use 256M) and related cache parameters such as `query_cache_limit' (we use 3M), as described on the above wiki page. (Beware also of per-connection limit times the number of max allowed connections.) A nice tool that can help you in tweaking DB server parameters according to your concrete DB state and your real-life traffic is `mysqltuner'. I've been using also `mysqlreport' and `tuning-primer.sh' with good results. Best regards -- Tibor Simko
