Some users' code is causing MySQL's memory use to explode. By the time we reach about 200 simultaneous connections, the MySQL server is using 8GB of virtual memory, and then falls over (the machine is an AlphaServer ES45 with 8 GB of physical memory, and 16 GB of swap, although processes are constrained to a total of 8GB of virtual memory in the OS) The versions of MySQL I have found this behaviour on are 4.0.14 and 4.0.17
I'm presuming that I have some configuration variable wrong somewhere, but I can't work out what it is. Here's our .cnf file for the instance:
[mysqld_safe] err-log=/mysql/log_3365/err.log
[mysqladmin] socket=/mysql/data_3365/mysql_3365.sock port=3365
[mysqld] binlog_cache_size=32M datadir=/mysql/data_3365/databases interactive_timeout=2678200 key_buffer=1024M #log=/mysql/log_3365/query.log log_bin=/mysql/log_3365/bin.log log_slow_queries=/mysql/log_3365/slow.log log_warnings max_allowed_packet=16M max_binlog_size=2000M max_connections=1024 net_write_timeout=60 pid-file=/mysql/data_3365/mysql_3365.pid port=3365 query_cache_size=32M read_buffer_size=256K socket=/mysql/data_3365/mysql_3365.sock sort_buffer_size=2M table_cache=512 thread_cache_size=16 wait_timeout=2678200
# Replication options server_id=1
I've tried reducing sort_buffer_size to a pathetic 32K, and it makes no difference, so it's not that. Similarly, I've tried reducing max_allowed_packet, and that makes no difference.
The code in question is running a large number of compute jobs on a Linux cluster, and these jobs talk to the MySQL server both to inform a master control process what is going on, and secondly to store their results in it.
If I constrain the number of simultaneously running jobs to 20, then MySQL only grows to about 3.5 GB, 2.5 GB of which were allocated as soon as it started, so it looks like each connection is allocating around 50 MB inside MySQL, but I don't know where this is coming from. Surely each thread within the OS doesn't take 50 MB before it's allocated anything else?
Any ideas, including ways I can get MySQL to tell me more about what it's doing, would be most helpful. The query log, even with log-warnings on, does not tell us much.
Many thanks in advance...
Tim
-- Dr Tim Cutts Informatics Systems Group Wellcome Trust Sanger Institute Hinxton, Cambridge, CB10 1SA, UK
-- MySQL General Mailing List For list archives: http://lists.mysql.com/mysql To unsubscribe: http://lists.mysql.com/[EMAIL PROTECTED]
