Hello all,
I have a nightly script that executes a mysql batch file that takes
approximately 8 minutes to run. Unfortunately it makes my web server
unresponsive for all 8 minutes. I've tried ionice and nice to lower
the priority, but it still seems to consume all resources. The weird
thing is top
On Jan 18, 2012, at 11:58 AM, Eric Cope wrote:
Hello all,
I have a nightly script that executes a mysql batch file that takes
approximately 8 minutes to run. Unfortunately it makes my web server
unresponsive for all 8 minutes. I've tried ionice and nice to lower
the priority, but it still
good point. Its just a series of INSERTs (8000 or so), but the table
has a fulltext index, so its pretty slow (10 records/sec).
It sounds like I should split up the batch script, then load a few,
pause, load a few, pause, etc.
Any ideas?
Eric
On Wed, Jan 18, 2012 at 12:20 PM, Alex Dean
Damn, would have told you to use InnoDB, but InnoDB doesn't have fulltext.
MyISAM, as I learned today, locks an entire table during an insert, so
other records can't be inserted or updated while an insert is taking
place. So 8 minutes of constant inserts will cause some problems here.
Aye, best
From: Eric Cope eric.c...@gmail.com
good point. Its just a series of INSERTs (8000 or so), but the table
has a fulltext index, so its pretty slow (10 records/sec).
It sounds like I should split up the batch script, then load a few,
pause, load a few, pause, etc.
Run through 10 inserts, sleep
we had intended to migrate to Solr, but not this soon in the
development process. This may force our hand.
Eric
On Wed, Jan 18, 2012 at 12:51 PM, Matt Graham danceswithcr...@usa.net wrote:
From: Eric Cope eric.c...@gmail.com
good point. Its just a series of INSERTs (8000 or so), but the table
On Jan 18, 2012, at 1:30 PM, Eric Cope wrote:
good point. Its just a series of INSERTs (8000 or so), but the table
has a fulltext index, so its pretty slow (10 records/sec).
It sounds like I should split up the batch script, then load a few,
pause, load a few, pause, etc.
Any ideas?
Eric
Hi!
On Wed, Jan 18, 2012 at 1:27 PM, Alex Dean a...@crackpot.org wrote:
On Jan 18, 2012, at 1:30 PM, Eric Cope wrote:
good point. Its just a series of INSERTs (8000 or so), but the table
has a fulltext index, so its pretty slow (10 records/sec).
It sounds like I should split up the batch
I only have one column that is needed for the fulltext search. I think
it would be easier to integrate solr than to try and get MySQL to be
better. But this comes from someone who has never used solr.
So far, I have used csplit to split the file, insert one record, sleep
for 0.01s and repeat. Its
Full text is in innodb if you use a bleeding edge build. But I would
still recomend off loading. Untill the you can use better inserts
(load data in file) or split them up. MySQL does support sleep SELECT
SLEEP(2); for example. Run 100 then sleep 2 then run 100 and repeat
untill done. Slow for 30
10 matches
Mail list logo
