Lost connection to MySQL server - need help.

2013-10-12 Thread Jørn Dahl-Stamnes
Hello,

I got a strange problem related to a production server. It has been working OK 
for months, but yesterday it start to fail. There are several batch scripts 
using the database in addition to a web application using it.

The php scripts running in batch mode began to get:

mysql_connect(): Lost connection to MySQL server at 'reading initial 
communication packet', system error: 111

I stopped the server and restarted it and everything seems to work OK for 
hours but when the load start to increase, the errors begin to appear again.

Today I noticed that after I starte phpMyAdmin and selected one of the 
databases, phpMyAdmin was hanging and the batch scripts began to fail again. 
Seems like the server does not handle much load anymore.


What's strange is the memory usage. The server is a quad core cpu with 48 Gb 
memory, where 28 Gb is allocated to innodb (we mostly use innodb). But when 
using top command, I noticed this:

VIRT: 33.9g
RES: 9.4g
SWAP: 23g

at this time over 11G memory is free. vm.swappiness is set to 0. I find it 
strange that the server is not able to use physical memory but use swap 
instead. The amount of cpu time used for swapping is rather high during sql 
queries. The amount of RESident memory may increase slowly over time but very 
slowly (it can take hours before it increase to 20+ Gb).

[PS: I also got a MySQL server running at a dedicated host at home, where the 
it seem to use the memory as I except it to use:

  PID USER  PR  NI  VIRT  RES  SHR S %CPU %MEMTIME+  SWAP DATA COMMAND
 1462 mysql 20   0 30.0g  27g 3900 S  0.3 87.3   2633:14 844m  29g mysqld
]


I would like to have some suggestions what I can do to solve this problem.
I have google'd it but found nothing that seem to solve my case.

Server:
  OS: Debian 6
  MySQL: 5.1.61-0+squeeze1

my.cnf:
#
# The MySQL database server configuration file.
#

[client]
port= 3306
socket  = /var/run/mysqld/mysqld.sock

# Here is entries for some specific programs
# The following values assume you have at least 32M ram

# This was formally known as [safe_mysqld].
[mysqld_safe]
socket  = /var/run/mysqld/mysqld.sock
nice= 0

[mysqld]
#
# * Basic Settings
#
user= mysql
pid-file= /var/run/mysqld/mysqld.pid
socket  = /var/run/mysqld/mysqld.sock
port= 3306
basedir = /usr
datadir = /database/mysql
tmpdir  = /tmp
language= /usr/share/mysql/english
skip-external-locking
#
# Instead of skip-networking the default is now to listen only on
# localhost which is more compatible and is not less secure.
bind-address= 127.0.0.1
## All applications use 127.0.0.1 when connectiong to the db.

#
# * Fine Tuning
#
#key_buffer = 16M
max_allowed_packet  = 64M
thread_stack= 192K
#thread_cache_size   = 8

# This replaces the startup script and checks MyISAM tables if needed
# the first time they are touched
myisam-recover = BACKUP
#
# * Query Cache Configuration
#
query_cache_limit   = 1M

#
# The following can be used as easy to replay backup logs or for replication.
# note: if you are setting up a replication slave, see README.Debian about
#   other settings you may need to change.
#server-id  = 1
#log_bin= /var/log/mysql/mysql-bin.log
expire_logs_days= 10
max_binlog_size = 100M
#binlog_do_db   = include_database_name
#binlog_ignore_db   = include_database_name
#
# * InnoDB
#
# InnoDB is enabled by default with a 10MB datafile in /var/lib/mysql/.
# Read the manual for more InnoDB related options. There are many!

thread_cache_size = 192
table_cache = 768
## key_buffer = 64M
## sort_buffer_size = 256K
## read_buffer_size = 256K
## read_rnd_buffer_size = 256K
tmp_table_size=32M
max_heap_table_size=32M
query_cache_size=128M
query_cache_type=2

innodb_open_files=1000
innodb_buffer_pool_size = 28G
innodb_additional_mem_pool_size = 8M
innodb_flush_log_at_trx_commit = 1
innodb_support_xa = 0
innodb_lock_wait_timeout = 50
## innodb_flush_method=O_DIRECT
innodb_log_files_in_group = 2
## innodb_log_file_size = 128M
innodb_log_buffer_size = 8M
innodb_thread_concurrency = 14
innodb_file_per_table

max_connections = 100
binlog_cache_size   = 1M
sort_buffer_size= 16M
join_buffer_size= 16M
ft_min_word_len = 1
ft_max_word_len = 84
ft_stopword_file= ''
default_table_type  = InnoDB
key_buffer  = 2G
read_buffer_size= 2M
read_rnd_buffer_size= 16M
bulk_insert_buffer_size = 64M
myisam_sort_buffer_size = 128M
myisam_max_sort_file_size   = 10G
myisam_max_extra_sort_file_size = 10G
myisam_repair_threads   = 1
myisam_recover

[mysqldump]
quick
quote-names
max_allowed_packet = 16M

[mysql]
#no-auto-rehash # faster start of mysql but no tab 

Re: Lost connection to MySQL server - need help.

2013-10-12 Thread nixofortune

You might want to comment

bind-address= 127.0.0.1

in your my.cnf and restart mysql server.



On 12/10/13 10:49, Jørn Dahl-Stamnes wrote:

Hello,

I got a strange problem related to a production server. It has been working OK
for months, but yesterday it start to fail. There are several batch scripts
using the database in addition to a web application using it.

The php scripts running in batch mode began to get:

mysql_connect(): Lost connection to MySQL server at 'reading initial
communication packet', system error: 111

I stopped the server and restarted it and everything seems to work OK for
hours but when the load start to increase, the errors begin to appear again.

Today I noticed that after I starte phpMyAdmin and selected one of the
databases, phpMyAdmin was hanging and the batch scripts began to fail again.
Seems like the server does not handle much load anymore.


What's strange is the memory usage. The server is a quad core cpu with 48 Gb
memory, where 28 Gb is allocated to innodb (we mostly use innodb). But when
using top command, I noticed this:

VIRT: 33.9g
RES: 9.4g
SWAP: 23g

at this time over 11G memory is free. vm.swappiness is set to 0. I find it
strange that the server is not able to use physical memory but use swap
instead. The amount of cpu time used for swapping is rather high during sql
queries. The amount of RESident memory may increase slowly over time but very
slowly (it can take hours before it increase to 20+ Gb).

[PS: I also got a MySQL server running at a dedicated host at home, where the
it seem to use the memory as I except it to use:

   PID USER  PR  NI  VIRT  RES  SHR S %CPU %MEMTIME+  SWAP DATA COMMAND
  1462 mysql 20   0 30.0g  27g 3900 S  0.3 87.3   2633:14 844m  29g mysqld
]


I would like to have some suggestions what I can do to solve this problem.
I have google'd it but found nothing that seem to solve my case.

Server:
   OS: Debian 6
   MySQL: 5.1.61-0+squeeze1

my.cnf:
#
# The MySQL database server configuration file.
#

[client]
port= 3306
socket  = /var/run/mysqld/mysqld.sock

# Here is entries for some specific programs
# The following values assume you have at least 32M ram

# This was formally known as [safe_mysqld].
[mysqld_safe]
socket  = /var/run/mysqld/mysqld.sock
nice= 0

[mysqld]
#
# * Basic Settings
#
user= mysql
pid-file= /var/run/mysqld/mysqld.pid
socket  = /var/run/mysqld/mysqld.sock
port= 3306
basedir = /usr
datadir = /database/mysql
tmpdir  = /tmp
language= /usr/share/mysql/english
skip-external-locking
#
# Instead of skip-networking the default is now to listen only on
# localhost which is more compatible and is not less secure.
bind-address= 127.0.0.1
## All applications use 127.0.0.1 when connectiong to the db.

#
# * Fine Tuning
#
#key_buffer = 16M
max_allowed_packet  = 64M
thread_stack= 192K
#thread_cache_size   = 8

# This replaces the startup script and checks MyISAM tables if needed
# the first time they are touched
myisam-recover = BACKUP
#
# * Query Cache Configuration
#
query_cache_limit   = 1M

#
# The following can be used as easy to replay backup logs or for replication.
# note: if you are setting up a replication slave, see README.Debian about
#   other settings you may need to change.
#server-id  = 1
#log_bin= /var/log/mysql/mysql-bin.log
expire_logs_days= 10
max_binlog_size = 100M
#binlog_do_db   = include_database_name
#binlog_ignore_db   = include_database_name
#
# * InnoDB
#
# InnoDB is enabled by default with a 10MB datafile in /var/lib/mysql/.
# Read the manual for more InnoDB related options. There are many!

thread_cache_size = 192
table_cache = 768
## key_buffer = 64M
## sort_buffer_size = 256K
## read_buffer_size = 256K
## read_rnd_buffer_size = 256K
tmp_table_size=32M
max_heap_table_size=32M
query_cache_size=128M
query_cache_type=2

innodb_open_files=1000
innodb_buffer_pool_size = 28G
innodb_additional_mem_pool_size = 8M
innodb_flush_log_at_trx_commit = 1
innodb_support_xa = 0
innodb_lock_wait_timeout = 50
## innodb_flush_method=O_DIRECT
innodb_log_files_in_group = 2
## innodb_log_file_size = 128M
innodb_log_buffer_size = 8M
innodb_thread_concurrency = 14
innodb_file_per_table

max_connections = 100
binlog_cache_size   = 1M
sort_buffer_size= 16M
join_buffer_size= 16M
ft_min_word_len = 1
ft_max_word_len = 84
ft_stopword_file= ''
default_table_type  = InnoDB
key_buffer  = 2G
read_buffer_size= 2M
read_rnd_buffer_size= 16M
bulk_insert_buffer_size = 64M
myisam_sort_buffer_size = 128M
myisam_max_sort_file_size   = 10G
myisam_max_extra_sort_file_size = 10G
myisam_repair_threads   

Re: Lost connection to MySQL server - need help.

2013-10-12 Thread Jørn Dahl-Stamnes
On Saturday 12 October 2013 12:01, nixofortune wrote:
 You might want to comment

 bind-address= 127.0.0.1

 in your my.cnf and restart mysql server.

It does not explain why it works under low load and not under high load.

However, I seem to have found something. When I started phpMyAdmin and 
selected one of the database, the server went away again and I found this 
in /var/log/syslog:

Oct 12 11:53:33 cebycny mysqld: 131012 11:53:33  InnoDB: Assertion failure in 
thread 140182892447488 in file ../../../storage/innobase/handler/ha_innodb.cc 
line
 8066
Oct 12 11:53:33 cebycny mysqld: InnoDB: Failing assertion: auto_inc  0
Oct 12 11:53:33 cebycny mysqld: InnoDB: We intentionally generate a memory 
trap.
Oct 12 11:53:33 cebycny mysqld: InnoDB: Submit a detailed bug report to 
http://bugs.mysql.com.
Oct 12 11:53:33 cebycny mysqld: InnoDB: If you get repeated assertion failures 
or crashes, even
Oct 12 11:53:33 cebycny mysqld: InnoDB: immediately after the mysqld startup, 
there may be
Oct 12 11:53:33 cebycny mysqld: InnoDB: corruption in the InnoDB tablespace. 
Please refer to
Oct 12 11:53:33 cebycny mysqld: InnoDB: 
http://dev.mysql.com/doc/refman/5.1/en/forcing-innodb-recovery.html
Oct 12 11:53:33 cebycny mysqld: InnoDB: about forcing recovery.
Oct 12 11:53:33 cebycny mysqld: 09:53:33 UTC - mysqld got signal 6 ;
Oct 12 11:53:33 cebycny mysqld: This could be because you hit a bug. It is 
also possible that this binary
Oct 12 11:53:33 cebycny mysqld: or one of the libraries it was linked against 
is corrupt, improperly built,
Oct 12 11:53:33 cebycny mysqld: or misconfigured. This error can also be 
caused by malfunctioning hardware.
Oct 12 11:53:33 cebycny mysqld: We will try our best to scrape up some info 
that will hopefully help
Oct 12 11:53:33 cebycny mysqld: diagnose the problem, but since we have 
already crashed,
Oct 12 11:53:33 cebycny mysqld: something is definitely wrong and this may 
fail.
Oct 12 11:53:33 cebycny mysqld:
Oct 12 11:53:33 cebycny mysqld: key_buffer_size=2147483648
Oct 12 11:53:33 cebycny mysqld: read_buffer_size=2097152
Oct 12 11:53:33 cebycny mysqld: max_used_connections=8
Oct 12 11:53:33 cebycny mysqld: max_threads=100
Oct 12 11:53:33 cebycny mysqld: thread_count=2
Oct 12 11:53:33 cebycny mysqld: connection_count=2
Oct 12 11:53:33 cebycny mysqld: It is possible that mysqld could use up to
Oct 12 11:53:33 cebycny mysqld: key_buffer_size + (read_buffer_size + 
sort_buffer_size)*max_threads = 3941387 K  bytes of memory
Oct 12 11:53:33 cebycny mysqld: Hope that's ok; if not, decrease some 
variables in the equation.
Oct 12 11:53:33 cebycny mysqld:
Oct 12 11:53:33 cebycny mysqld: Thread pointer: 0x7f7f1bf997c0
Oct 12 11:53:33 cebycny mysqld: Attempting backtrace. You can use the 
following information to find out
Oct 12 11:53:33 cebycny mysqld: where mysqld died. If you see no messages 
after this, something went
Oct 12 11:53:33 cebycny mysqld: terribly wrong...
Oct 12 11:53:33 cebycny mysqld: stack_bottom = 7f7edf81fe88 thread_stack 
0x3
Oct 12 11:53:33 cebycny mysqld: /usr/sbin/mysqld(my_print_stacktrace+0x29) 
[0x7f7edff62b59]
Oct 12 11:53:33 cebycny mysqld: /usr/sbin/mysqld(handle_fatal_signal+0x483) 
[0x7f7edfd774a3]
Oct 12 11:53:33 cebycny mysqld: /lib/libpthread.so.0(+0xeff0) [0x7f7edf4c9ff0]
Oct 12 11:53:33 cebycny mysqld: /lib/libc.so.6(gsignal+0x35) [0x7f7eddf6c1b5]
Oct 12 11:53:33 cebycny mysqld: /lib/libc.so.6(abort+0x180) [0x7f7eddf6efc0]
Oct 12 11:53:33 cebycny 
mysqld: /usr/sbin/mysqld(ha_innobase::innobase_peek_autoinc()+0x8f) 
[0x7f7edfe1fa2f]
Oct 12 11:53:33 cebycny 
mysqld: /usr/sbin/mysqld(ha_innobase::info_low(unsigned int, bool)+0x18f) 
[0x7f7edfe2524f]
Oct 12 11:53:33 cebycny 
mysqld: 
/usr/sbin/mysqld(ha_innobase::update_create_info(st_ha_create_information*)+0x29)
 
[0x7f7edfe256b9]
Oct 12 11:53:33 cebycny mysqld: /usr/sbin/mysqld(+0x49e3dc) [0x7f7edfd953dc]
Oct 12 11:53:33 cebycny mysqld: /usr/sbin/mysqld(mysqld_show_create(THD*, 
TABLE_LIST*)+0x7a8) [0x7f7edfd9d388]
Oct 12 11:53:33 cebycny 
mysqld: /usr/sbin/mysqld(mysql_execute_command(THD*)+0x184a) [0x7f7edfc7cb0a]
Oct 12 11:53:33 cebycny mysqld: /usr/sbin/mysqld(mysql_parse(THD*, char*, 
unsigned int, char const**)+0x3fb) [0x7f7edfc80dbb]
Oct 12 11:53:33 cebycny 
mysqld: /usr/sbin/mysqld(dispatch_command(enum_server_command, THD*, char*, 
unsigned int)+0x115a) [0x7f7edfc81f2a]
Oct 12 11:53:33 cebycny mysqld: /usr/sbin/mysqld(do_command(THD*)+0xea) 
[0x7f7edfc8285a]
Oct 12 11:53:33 cebycny mysqld: /usr/sbin/mysqld(handle_one_connection+0x235) 
[0x7f7edfc74435]
Oct 12 11:53:33 cebycny mysqld: /lib/libpthread.so.0(+0x68ca) [0x7f7edf4c18ca]
Oct 12 11:53:33 cebycny mysqld: /lib/libc.so.6(clone+0x6d) [0x7f7ede00992d]
Oct 12 11:53:33 cebycny mysqld:
Oct 12 11:53:33 cebycny mysqld: Trying to get some variables.
Oct 12 11:53:33 cebycny mysqld: Some pointers may be invalid and cause the 
dump to abort.
Oct 12 11:53:33 cebycny mysqld: Query (7f7f1c0dcbc0): SHOW CREATE TABLE 
`calculation`
Oct 12 

Re: Lost connection to MySQL server - need help.

2013-10-12 Thread Andrew Moore
Could be a crash related to innodb data dictionary being out of sync. Could
be a bug.

http://bugs.mysql.com/bug.php?id=55277
On 12 Oct 2013 11:21, Jørn Dahl-Stamnes sq...@dahl-stamnes.net wrote:

 On Saturday 12 October 2013 12:01, nixofortune wrote:
  You might want to comment
 
  bind-address= 127.0.0.1
 
  in your my.cnf and restart mysql server.

 It does not explain why it works under low load and not under high load.

 However, I seem to have found something. When I started phpMyAdmin and
 selected one of the database, the server went away again and I found this
 in /var/log/syslog:

 Oct 12 11:53:33 cebycny mysqld: 131012 11:53:33  InnoDB: Assertion failure
 in
 thread 140182892447488 in file
 ../../../storage/innobase/handler/ha_innodb.cc
 line
  8066
 Oct 12 11:53:33 cebycny mysqld: InnoDB: Failing assertion: auto_inc  0
 Oct 12 11:53:33 cebycny mysqld: InnoDB: We intentionally generate a memory
 trap.
 Oct 12 11:53:33 cebycny mysqld: InnoDB: Submit a detailed bug report to
 http://bugs.mysql.com.
 Oct 12 11:53:33 cebycny mysqld: InnoDB: If you get repeated assertion
 failures
 or crashes, even
 Oct 12 11:53:33 cebycny mysqld: InnoDB: immediately after the mysqld
 startup,
 there may be
 Oct 12 11:53:33 cebycny mysqld: InnoDB: corruption in the InnoDB
 tablespace.
 Please refer to
 Oct 12 11:53:33 cebycny mysqld: InnoDB:
 http://dev.mysql.com/doc/refman/5.1/en/forcing-innodb-recovery.html
 Oct 12 11:53:33 cebycny mysqld: InnoDB: about forcing recovery.
 Oct 12 11:53:33 cebycny mysqld: 09:53:33 UTC - mysqld got signal 6 ;
 Oct 12 11:53:33 cebycny mysqld: This could be because you hit a bug. It is
 also possible that this binary
 Oct 12 11:53:33 cebycny mysqld: or one of the libraries it was linked
 against
 is corrupt, improperly built,
 Oct 12 11:53:33 cebycny mysqld: or misconfigured. This error can also be
 caused by malfunctioning hardware.
 Oct 12 11:53:33 cebycny mysqld: We will try our best to scrape up some info
 that will hopefully help
 Oct 12 11:53:33 cebycny mysqld: diagnose the problem, but since we have
 already crashed,
 Oct 12 11:53:33 cebycny mysqld: something is definitely wrong and this may
 fail.
 Oct 12 11:53:33 cebycny mysqld:
 Oct 12 11:53:33 cebycny mysqld: key_buffer_size=2147483648
 Oct 12 11:53:33 cebycny mysqld: read_buffer_size=2097152
 Oct 12 11:53:33 cebycny mysqld: max_used_connections=8
 Oct 12 11:53:33 cebycny mysqld: max_threads=100
 Oct 12 11:53:33 cebycny mysqld: thread_count=2
 Oct 12 11:53:33 cebycny mysqld: connection_count=2
 Oct 12 11:53:33 cebycny mysqld: It is possible that mysqld could use up to
 Oct 12 11:53:33 cebycny mysqld: key_buffer_size + (read_buffer_size +
 sort_buffer_size)*max_threads = 3941387 K  bytes of memory
 Oct 12 11:53:33 cebycny mysqld: Hope that's ok; if not, decrease some
 variables in the equation.
 Oct 12 11:53:33 cebycny mysqld:
 Oct 12 11:53:33 cebycny mysqld: Thread pointer: 0x7f7f1bf997c0
 Oct 12 11:53:33 cebycny mysqld: Attempting backtrace. You can use the
 following information to find out
 Oct 12 11:53:33 cebycny mysqld: where mysqld died. If you see no messages
 after this, something went
 Oct 12 11:53:33 cebycny mysqld: terribly wrong...
 Oct 12 11:53:33 cebycny mysqld: stack_bottom = 7f7edf81fe88 thread_stack
 0x3
 Oct 12 11:53:33 cebycny mysqld: /usr/sbin/mysqld(my_print_stacktrace+0x29)
 [0x7f7edff62b59]
 Oct 12 11:53:33 cebycny mysqld: /usr/sbin/mysqld(handle_fatal_signal+0x483)
 [0x7f7edfd774a3]
 Oct 12 11:53:33 cebycny mysqld: /lib/libpthread.so.0(+0xeff0)
 [0x7f7edf4c9ff0]
 Oct 12 11:53:33 cebycny mysqld: /lib/libc.so.6(gsignal+0x35)
 [0x7f7eddf6c1b5]
 Oct 12 11:53:33 cebycny mysqld: /lib/libc.so.6(abort+0x180)
 [0x7f7eddf6efc0]
 Oct 12 11:53:33 cebycny
 mysqld: /usr/sbin/mysqld(ha_innobase::innobase_peek_autoinc()+0x8f)
 [0x7f7edfe1fa2f]
 Oct 12 11:53:33 cebycny
 mysqld: /usr/sbin/mysqld(ha_innobase::info_low(unsigned int, bool)+0x18f)
 [0x7f7edfe2524f]
 Oct 12 11:53:33 cebycny
 mysqld:
 /usr/sbin/mysqld(ha_innobase::update_create_info(st_ha_create_information*)+0x29)
 [0x7f7edfe256b9]
 Oct 12 11:53:33 cebycny mysqld: /usr/sbin/mysqld(+0x49e3dc)
 [0x7f7edfd953dc]
 Oct 12 11:53:33 cebycny mysqld: /usr/sbin/mysqld(mysqld_show_create(THD*,
 TABLE_LIST*)+0x7a8) [0x7f7edfd9d388]
 Oct 12 11:53:33 cebycny
 mysqld: /usr/sbin/mysqld(mysql_execute_command(THD*)+0x184a)
 [0x7f7edfc7cb0a]
 Oct 12 11:53:33 cebycny mysqld: /usr/sbin/mysqld(mysql_parse(THD*, char*,
 unsigned int, char const**)+0x3fb) [0x7f7edfc80dbb]
 Oct 12 11:53:33 cebycny
 mysqld: /usr/sbin/mysqld(dispatch_command(enum_server_command, THD*, char*,
 unsigned int)+0x115a) [0x7f7edfc81f2a]
 Oct 12 11:53:33 cebycny mysqld: /usr/sbin/mysqld(do_command(THD*)+0xea)
 [0x7f7edfc8285a]
 Oct 12 11:53:33 cebycny mysqld:
 /usr/sbin/mysqld(handle_one_connection+0x235)
 [0x7f7edfc74435]
 Oct 12 11:53:33 cebycny mysqld: /lib/libpthread.so.0(+0x68ca)
 [0x7f7edf4c18ca]
 Oct 12 11:53:33 cebycny mysqld: /lib/libc.so.6(clone+0x6d) [0x7f7ede00992d]
 Oct 12 

Re: Lost connection to MySQL server - need help.

2013-10-12 Thread Jørn Dahl-Stamnes
On Saturday 12 October 2013 13:07, Andrew Moore wrote:
 Could be a crash related to innodb data dictionary being out of sync. Could
 be a bug.

Seems like a bug yes. However, we had a strange situation yesterday when we 
had several processes in the state copying to tmp table (if i remember the 
exact phrase). After witing 2 seconds, I restarted the server. It seemed 
to work OK until the backup started.

Perhaps we should restore the database that I suspect cause this, in order to 
rebuild the complete database.

-- 
Jørn Dahl-Stamnes
homepage: http://photo.dahl-stamnes.net/

--
MySQL General Mailing List
For list archives: http://lists.mysql.com/mysql
To unsubscribe:http://lists.mysql.com/mysql



Re: Lost connection to MySQL server - need help.

2013-10-12 Thread Reindl Harald


Am 12.10.2013 17:02, schrieb Jørn Dahl-Stamnes:
 On Saturday 12 October 2013 13:07, Andrew Moore wrote:
 Could be a crash related to innodb data dictionary being out of sync. Could
 be a bug.
 
 Seems like a bug yes. However, we had a strange situation yesterday when we 
 had several processes in the state copying to tmp table (if i remember the 
 exact phrase). After witing 2 seconds, I restarted the server. It seemed 
 to work OK until the backup started

so someone did optimize table on a large table
you do yourself not a favour restarting the server in such a moment



signature.asc
Description: OpenPGP digital signature


Re: Lost connection to MySQL server - need help.

2013-10-12 Thread Jørn Dahl-Stamnes
On Saturday 12 October 2013 17:36, Reindl Harald wrote:
 so someone did optimize table on a large table
 you do yourself not a favour restarting the server in such a moment

7 hours before the server was shut down, we did a alter table to add a primary 
key to a table that is read-only from the web application.

-- 
Jørn Dahl-Stamnes
homepage: http://photo.dahl-stamnes.net/

--
MySQL General Mailing List
For list archives: http://lists.mysql.com/mysql
To unsubscribe:http://lists.mysql.com/mysql



Re: Lost connection to MySQL server - need help.

2013-10-12 Thread Reindl Harald


Am 12.10.2013 19:45, schrieb Jørn Dahl-Stamnes:
 On Saturday 12 October 2013 17:36, Reindl Harald wrote:
 so someone did optimize table on a large table
 you do yourself not a favour restarting the server in such a moment
 
 7 hours before the server was shut down, we did a alter table to add a 
 primary 
 key to a table that is read-only from the web application.

which means the table is most likely completly copied
in a temp file and depending on the table size this
takes time - you killed the alter table i guess



signature.asc
Description: OpenPGP digital signature


Re: Lost connection to MySQL server - need help.

2013-10-12 Thread Chris McKeever
We had a similar issue a bit back - and although it sounds similar - based
on your followups it probably isnt, but will just toss this out there
anyhows.  We were experiencing connection timeouts when load would ramp up.
 Doing some digging we learned that our firewall between the servers
bandwidth would get consumed by a large wordpress load - and this in
essence backed up the rest of the requests until they timed out.

We fixed that load issue which reduced the data passing through and have
expereinced a significant performance boost in our app let alone reduction
of these timeout issues




On Sat, Oct 12, 2013 at 12:56 PM, Reindl Harald h.rei...@thelounge.netwrote:



 Am 12.10.2013 19:45, schrieb Jørn Dahl-Stamnes:
  On Saturday 12 October 2013 17:36, Reindl Harald wrote:
  so someone did optimize table on a large table
  you do yourself not a favour restarting the server in such a moment
 
  7 hours before the server was shut down, we did a alter table to add a
 primary
  key to a table that is read-only from the web application.

 which means the table is most likely completly copied
 in a temp file and depending on the table size this
 takes time - you killed the alter table i guess




Re: Lost connection to MySQL server - need help.

2013-10-12 Thread Reindl Harald
sounds like a scheduler issue

did you try deadline?
http://en.wikipedia.org/wiki/Deadline_scheduler

on Linux systems pass elevator=deadline as kernel param

Am 12.10.2013 20:58, schrieb Chris McKeever:
 We had a similar issue a bit back - and although it sounds similar - based
 on your followups it probably isnt, but will just toss this out there
 anyhows.  We were experiencing connection timeouts when load would ramp up.
  Doing some digging we learned that our firewall between the servers
 bandwidth would get consumed by a large wordpress load - and this in
 essence backed up the rest of the requests until they timed out.
 
 We fixed that load issue which reduced the data passing through and have
 expereinced a significant performance boost in our app let alone reduction
 of these timeout issues
 
 On Sat, Oct 12, 2013 at 12:56 PM, Reindl Harald h.rei...@thelounge.netwrote:
 


 Am 12.10.2013 19:45, schrieb Jørn Dahl-Stamnes:
 On Saturday 12 October 2013 17:36, Reindl Harald wrote:
 so someone did optimize table on a large table
 you do yourself not a favour restarting the server in such a moment

 7 hours before the server was shut down, we did a alter table to add a
 primary
 key to a table that is read-only from the web application.

 which means the table is most likely completly copied
 in a temp file and depending on the table size this
 takes time - you killed the alter table i guess




signature.asc
Description: OpenPGP digital signature