[Process:]
Importing
delimited text files from a Windows based server to a MySQL 5.1.41 instance
(multiple databases) on a single Ubuntu 10.04.2 host.
The
process is initiated on the Windows server via the MySQL exe using ‘load data
local infile’.
There are
20 databases total and we import 15 files per database – 1 file per
table.
All tables
use the MyISAM engine.
Prior to
each import, we truncate the destination tables.
All
processing is done sequentially.
[Issues:]
The process
will run fine for about 2 weeks then continues to increase significantly in
overall processing time. For example, we’ll see a 35 minute run-time for
2 weeks, then the next day its 55 minutes, then 80 minutes, etc, then without
any changes, it will drop back to 35 after about a week (length of times
vary.) It does not seem to be the result of any other processing
requirements as the server is basically dedicated to MySQL only and there are
no manually added CRON tasks.
Even the
truncate table steps appear to be affected.
An
comparison of ‘show status’ on a normal day vs a slow day does not appear to
show any major issues other than a high volume of aborted_connects but this
variable
seems to increase regularly, outside of the actual import processing window.
Not sure if
this is some sort of MySQL resource buildup or something related to the OS.
Any input
would be greatly appreciated.
