Yes, the article did lack a lot of methodology information.
Since you already have things separated and it seems your data isn't "ciritical", meaning losing it would be a pain as opposed to
life threatening, you can turn off journaling for those file systems. You may actually want to look at other file system for those
volumes too.
I'm not going to recommend InnoDB, it doesn't seem like you need transactions and clearly you are outside the physicial size where
InndoDB performs well.
This might be a silly question, but did you max out the myisam_sort_buffer_size and key_buffer_size settings? Both can go up to 4GB.
After indexes are created you can drop them back down since those large numbers are "usually" unnecessary for daily use.
You can also turn off indexing when loading the data, then turn it back on when the data is loaded (if you haven't already). But
I've had some funky things happen when I've done this with large datasets (500 million records) and auto increment fields.
----- Original Message -----
From: "Chris Kantarjiev" <[EMAIL PROTECTED]>
To: <[EMAIL PROTECTED]>; <[EMAIL PROTECTED]>; <mysql@lists.mysql.com>
Sent: Friday, March 09, 2007 12:09 PM
Subject: Re: what journal options should I use on linux?
Brent,
Thanks for your response.
Enabling journaling isn't going to halve your performance.
I was careful to say "write speed", not "performance".
I already have my data and index files on separate drives (raid
volumes, actually, each made up of complete drives). What I see
is that the index drive is being clobbered during table creation,
because mysql can't keep it all in memory. This is a long standing
problem with MyISAM files, where the index code isn't 64-bit safe.
Yes, 64-bit. This is a quad-processor opteron with 16GB of ram.
The index file is 15GB these days, so even if My ISAM *could*
hold more than about 3GB of index in its data structures, it
probably wouldn't all fit in memory.
Did I mention that this is a "big data" problem?
Please don't tell me to use InnoDB. It's much too slow for this purpose.
Here is an interesting article to read on ext3 journaling overhead.
http://www-128.ibm.com/developerworks/library/l-fs8.html
Interesting, if only to show how dangerous it is to publish
results that aren't understood. The author doesn't say anything
about testing methodology, so I have no idea whether or not to
trust the results. 16MB files are toys; they easily fit completely
in memory and Linux makes it difficult to clear the buffer cache
between runs. Was the machine rebooted between every test? When
he runs these tests again with files that are bigger than available
RAM, I'll be a lot more interested.
--
MySQL General Mailing List
For list archives: http://lists.mysql.com/mysql
To unsubscribe: http://lists.mysql.com/[EMAIL PROTECTED]
