Odd MySQL performance behaviour

2011-07-18 Thread A F
[Process:]
 
Importing
delimited text files from a Windows based server to a MySQL 5.1.41 instance
(multiple databases) on a single Ubuntu 10.04.2 host. 
 The
process is initiated on the Windows server via the MySQL exe using ‘load data
local infile’.
There are
20 databases total and we import 15 files per database – 1 file per
table.  
All tables
use the MyISAM engine.
Prior to
each import, we truncate the destination tables.
All
processing is done sequentially.
 
[Issues:]
 
The process
will run fine for about 2 weeks then continues to increase significantly in
overall processing time.  For example, we’ll see a 35 minute run-time for
2 weeks, then the next day its 55 minutes, then 80 minutes, etc, then without
any changes, it will drop back to 35 after about a week (length of times
vary.)  It does not seem to be the result of any other processing
requirements as the server is basically dedicated to MySQL only and there are
no manually added CRON tasks.
Even the
truncate table steps appear to be affected.
 
An
comparison of ‘show status’ on a normal day vs a slow day does not appear to
show any major issues other than a high volume of aborted_connects but this 
variable
seems to increase regularly, outside of the actual import processing window.
 
Not sure if
this is some sort of MySQL resource buildup or something related to the OS.
 
Any input
would be greatly appreciated.

Re: Odd MySQL performance behaviour

2011-07-18 Thread walter harms
maybe its is obvoius but

did you look at the statistics ?
did you try optimize table ?

re,
 wh

Am 18.07.2011 18:40, schrieb A F:
 [Process:]
  
 Importing
 delimited text files from a Windows based server to a MySQL 5.1.41 instance
 (multiple databases) on a single Ubuntu 10.04.2 host. 
  The
 process is initiated on the Windows server via the MySQL exe using ‘load data
 local infile’.
 There are
 20 databases total and we import 15 files per database – 1 file per
 table.  
 All tables
 use the MyISAM engine.
 Prior to
 each import, we truncate the destination tables.
 All
 processing is done sequentially.
  
 [Issues:]
  
 The process
 will run fine for about 2 weeks then continues to increase significantly in
 overall processing time.  For example, we’ll see a 35 minute run-time for
 2 weeks, then the next day its 55 minutes, then 80 minutes, etc, then without
 any changes, it will drop back to 35 after about a week (length of times
 vary.)  It does not seem to be the result of any other processing
 requirements as the server is basically dedicated to MySQL only and there are
 no manually added CRON tasks.
 Even the
 truncate table steps appear to be affected.
  
 An
 comparison of ‘show status’ on a normal day vs a slow day does not appear to
 show any major issues other than a high volume of aborted_connects but this 
 variable
 seems to increase regularly, outside of the actual import processing window.
  
 Not sure if
 this is some sort of MySQL resource buildup or something related to the OS.
  
 Any input
 would be greatly appreciated.

-- 
MySQL General Mailing List
For list archives: http://lists.mysql.com/mysql
To unsubscribe:http://lists.mysql.com/mysql?unsub=arch...@jab.org