* Creigh Shank > Using an Apache/PHP/MySQL/Linux (Redhat 8.0) solution, PHPList, to create > an e-mailing list for our 5.6 million book club members. Unfortunately, > the import speed for importing records (at record number 150,000) > is about > 2,000 records per hour. We're running on the following: > > P4 (1.5 Ghz), 1.2 Gbytes RAM (650 Mbytes RAM Disk using ramfs), IDE drive > (7,200 rpm)
I miss a number in the calculation: how big is each record (or the total data file). If each record is 10MB, your speed is good, if it is 50 bytes, your speed is lousy. What method of import do you use? > So far we've moved the MySQL data files (var/lib/mysql), PHP /tmp and > upload directories and PHPlist web site files to RAM Disk (still just > testing - not yet dealing with data safety issues). With all of this > tuning we're still at only 2,000 records per hour for uploading. Do you mean you got no improvement moving to ramfs, or was it even worse before? Anyhow, something is wrong, unless your records are very big. You say 'uploading'? Is the import done remotely? What is the bandwidth? If putting everything on ramfs didn't improve anything, I would guess the network is the bottleneck. > We need to be at 100,000 records per hour 28 rows/sec, shouldn't be a problem, if the records are reasonably small, at least if you run the import locally on the server. > (uploading and sending seem to > run at about the same rate - we need to be able to send to all book club > members in the same week). sending? This looks more & more like a network issue... or maybe a "PHPList" issue? > Any suggestions? Give more info. :) -- Roger -- MySQL General Mailing List For list archives: http://lists.mysql.com/mysql To unsubscribe: http://lists.mysql.com/[EMAIL PROTECTED]
