innodb filesystem on software raid
I bought a supermicro 6013p-t for the 4 sata raid hard drives support. Unfortunately, it doesn't really have raid at all. So I'm forced to use software raid. What I'd like to use is fedora core 2 with an innodb filesystem on a software raid partition according to these instructions: http://dev.mysql.com/doc/mysql/en/InnoDB_Raw_Devices.html Has anybody done this? I'm not a linux expert and so I'm not sure exactly how to set this up. When I try to setup software raid in linux, I'm forced to pick a filesystem and its mount point as part of the process of creating a software raid partition. So this is the part that's stumping me. How do I create a software raid raw device only to use as an innodb filesystem? Is this possible? Or maybe this can't be done without hardware raid and I need to buy a new server? GNU's Parted software (http://www.gnu.org/software/parted/) has this to say regarding features, Supported disk labels: raw access (useful for RAID and LVM), ms-dos, etc. But I've been unable to create a raw software raid device with that as well. Scott Mueller AccelerateBiz Managed Hosting http://www.acceleratebiz.com http://www.acceleratebiz.com/ Phone: (800) 360-7360 Fax: (270) 778-3081
Re: innodb filesystem on software raid
I highly recommend simply using ext3 for your Linux setup. The 1 or 2 percent performance benefit that you may get from raw partitions is way outweighed by complexness of backups of the raw data. either way: First I would suggest you read the Linux RAID howto: http://www.tldp.org/HOWTO/Software-RAID-HOWTO.html Here are the basic steps: create a /etc/raidtab file for your array (probably md0) using a 32k or 64k chunk size (hint: man raidtab) run mkraid to initialize the new raid array (md0) (hint: man mkraid) if you want to use raw partitions: - #this is redhat/fedora specific add /dev/md0 to /etc/sysconfig/rawdevices (hint: man raw) add chown mysql:mysql /dev/raw/raw0 to /etc/init.d/rc.local if you want to use ext3: - mke2fs -j -T largefile4 /dev/md0 (hint: man mke2fs) --- Scott Mueller [EMAIL PROTECTED] wrote: I bought a supermicro 6013p-t for the 4 sata raid hard drives support. Unfortunately, it doesn't really have raid at all. So I'm forced to use software raid. What I'd like to use is fedora core 2 with an innodb filesystem on a software raid partition according to these instructions: http://dev.mysql.com/doc/mysql/en/InnoDB_Raw_Devices.html Has anybody done this? I'm not a linux expert and so I'm not sure exactly how to set this up. When I try to setup software raid in linux, I'm forced to pick a filesystem and its mount point as part of the process of creating a software raid partition. So this is the part that's stumping me. How do I create a software raid raw device only to use as an innodb filesystem? Is this possible? Or maybe this can't be done without hardware raid and I need to buy a new server? GNU's Parted software (http://www.gnu.org/software/parted/) has this to say regarding features, Supported disk labels: raw access (useful for RAID and LVM), ms-dos, etc. But I've been unable to create a raw software raid device with that as well. Scott Mueller AccelerateBiz Managed Hosting http://www.acceleratebiz.com http://www.acceleratebiz.com/ Phone: (800) 360-7360 Fax: (270) 778-3081 -- MySQL General Mailing List For list archives: http://lists.mysql.com/mysql To unsubscribe:http://lists.mysql.com/[EMAIL PROTECTED]
Re: InnoDB filesystem
On Fri, May 14, 2004 at 01:40:37PM +1000, Chris Nolan wrote: Jeremy Zawodny wrote: I think that the problem is that it's *not* a 64 bit OS. It's just an Intel 32bit box with 4GB of memory. And sine MySQL doesn't do PAE, it'll never see that extra memory. Didn't InnoDB gain PAE support on some platforms a little while ago? I think it may on Windows. That rings a very vague, distant bell. Jeremy -- Jeremy D. Zawodny | Perl, Web, MySQL, Linux Magazine, Yahoo! [EMAIL PROTECTED] | http://jeremy.zawodny.com/ [book] High Performance MySQL -- http://highperformancemysql.com/ -- MySQL General Mailing List For list archives: http://lists.mysql.com/mysql To unsubscribe:http://lists.mysql.com/[EMAIL PROTECTED]
Re: InnoDB filesystem
On 14 May 2004, at 1:14 am, Dathan Vance Pattishall wrote: -Original Message- From: Jeremy Zawodny [mailto:[EMAIL PROTECTED] Sent: Thursday, May 13, 2004 4:03 PM To: Dathan Vance Pattishall Cc: 'Tim Cutts'; 'MySQL List' Subject: Re: InnoDB filesystem On Thu, May 13, 2004 at 04:51:27PM -0700, Dathan Vance Pattishall wrote: I think that the problem is that it's *not* a 64 bit OS. It's just an Intel 32bit box with 4GB of memory. And sine MySQL doesn't do PAE, it'll never see that extra memory. Intel box with 4GB? It is possible with a patch like hugemem in Linux but 4GB should only be used 2^32 = 4GB. - The hugemem patch for instance allows you to use all 16 GB but at a performance penalty. I have 168 32-bit machines (IBM HS20 blades) which can take 8GB RAM each, although we don't have them configured with that much. So there are plenty of these machines available. I tend to agree though, that for MySQL, if you want lots of memory its better to go for a 64-bit platform. We've been using Alphas and Tru64 for MySQL databases for years. Tim -- MySQL General Mailing List For list archives: http://lists.mysql.com/mysql To unsubscribe:http://lists.mysql.com/[EMAIL PROTECTED]
Re: InnoDB filesystem
On 13 May 2004, at 3:34 pm, Dan Nelson wrote: Pros: performance and bypassing the filesystem cache. I believe most OSes support direct file access which either bypasses or minimizes cache effects, and InnoDB will enable it if possible. Solaris direct file I/O performance on UFS is within a couple percent of raw partitions, for example. This is all very well, but on some systems, surely the OS filesystem cache is a bonus, not a hindrance. Take for example a four-way X86 system with 16 GB RAM. I've seen people talk about such machines on this list. MySQL can't use all that memory itself, so it makes sense to allow the OS to cache as much disk space as possible in the memory that MySQL can't use directly? Tim -- MySQL General Mailing List For list archives: http://lists.mysql.com/mysql To unsubscribe:http://lists.mysql.com/[EMAIL PROTECTED]
InnoDB filesystem
I've been told that InnoDB on a raw partition is the fastest setup. To setup my system for this, could I create a partition called /innodb and adjust the my.cnf like this? innodb_data_home_dir = /innodb innodb_log_group_home_dir = /var/lib/mysql/ innodb_log_arch_dir = /var/lib/mysql/ I suppose that innodb_log_group_home_dir and innodb_log_arch_dir can not be on a raw disk? Any other tips, pros and cons? Thanks, Jacob -- MySQL General Mailing List For list archives: http://lists.mysql.com/mysql To unsubscribe:http://lists.mysql.com/[EMAIL PROTECTED]
Re: InnoDB filesystem
On Thu, May 13, 2004 at 11:00:17AM +0200, JFL wrote: I've been told that InnoDB on a raw partition is the fastest setup. Actually, you've been told that it's probably the fastest. To setup my system for this, could I create a partition called /innodb and adjust the my.cnf like this? innodb_data_home_dir = /innodb innodb_log_group_home_dir = /var/lib/mysql/ innodb_log_arch_dir = /var/lib/mysql/ Nope. I suppose that innodb_log_group_home_dir and innodb_log_arch_dir can not be on a raw disk? Check the InnoDB docs. They explain how to setup raw disk partitions. You'll be using device names, not mount points. Any other tips, pros and cons? Pros: performance and bypassing the filesystem cache. Cons: loss of transparency and limited backup options. Jeremy -- Jeremy D. Zawodny | Perl, Web, MySQL, Linux Magazine, Yahoo! [EMAIL PROTECTED] | http://jeremy.zawodny.com/ [book] High Performance MySQL -- http://highperformancemysql.com/ -- MySQL General Mailing List For list archives: http://lists.mysql.com/mysql To unsubscribe:http://lists.mysql.com/[EMAIL PROTECTED]
Re: InnoDB filesystem
I've been told that InnoDB on a raw partition is the fastest setup. Actually, you've been told that it's probably the fastest. Correct. Sorry :) Check the InnoDB docs. They explain how to setup raw disk partitions. You'll be using device names, not mount points. Thanks. I forgot to check the manual. Any other tips, pros and cons? Pros: performance and bypassing the filesystem cache. Cons: loss of transparency and limited backup options. What do you mean by transparency? Thanks for your help, Jacob -- MySQL General Mailing List For list archives: http://lists.mysql.com/mysql To unsubscribe:http://lists.mysql.com/[EMAIL PROTECTED]
Re: InnoDB filesystem
In the last episode (May 13), JFL said: I've been told that InnoDB on a raw partition is the fastest setup. Actually, you've been told that it's probably the fastest. Correct. Sorry :) Check the InnoDB docs. They explain how to setup raw disk partitions. You'll be using device names, not mount points. Thanks. I forgot to check the manual. Any other tips, pros and cons? Pros: performance and bypassing the filesystem cache. I believe most OSes support direct file access which either bypasses or minimizes cache effects, and InnoDB will enable it if possible. Solaris direct file I/O performance on UFS is within a couple percent of raw partitions, for example. Cons: loss of transparency and limited backup options. What do you mean by transparency? If you decide you need more space, you won't be able to simply extend the tablespace file; you'll need to either add another disk or resize your existing filesystems to free up space for another raw partition. And for backup, you'll either have to manually dd the data off the partition, or use mysqldump (or InnoDB hot backup) , since backup software will simply back up the device node without trying to read its contents. -- Dan Nelson [EMAIL PROTECTED] -- MySQL General Mailing List For list archives: http://lists.mysql.com/mysql To unsubscribe:http://lists.mysql.com/[EMAIL PROTECTED]
RE: InnoDB filesystem
-Original Message- From: Tim Cutts [mailto:[EMAIL PROTECTED] Sent: Thursday, May 13, 2004 7:11 AM To: MySQL List Subject: Re: InnoDB filesystem On 13 May 2004, at 3:34 pm, Dan Nelson wrote: Pros: performance and bypassing the filesystem cache. MySQL can't use all that memory itself, so it makes sense to allow the OS to cache as much disk space as possible in the memory that MySQL can't use directly? It depends, if your datafile is less then 16 GB then the system cache can help, but fill up the innodb_buffer_pool you'll get better performance. Think of innodb as being its own virtual filesystem. If you have 16GB it's probably a 64 bit OS, and mysql is available in 64 bit. Tim -- MySQL General Mailing List For list archives: http://lists.mysql.com/mysql To unsubscribe: http://lists.mysql.com/[EMAIL PROTECTED] -- MySQL General Mailing List For list archives: http://lists.mysql.com/mysql To unsubscribe:http://lists.mysql.com/[EMAIL PROTECTED]
Re: InnoDB filesystem
On Thu, May 13, 2004 at 04:51:27PM -0700, Dathan Vance Pattishall wrote: -Original Message- From: Tim Cutts [mailto:[EMAIL PROTECTED] Sent: Thursday, May 13, 2004 7:11 AM To: MySQL List Subject: Re: InnoDB filesystem On 13 May 2004, at 3:34 pm, Dan Nelson wrote: Pros: performance and bypassing the filesystem cache. MySQL can't use all that memory itself, so it makes sense to allow the OS to cache as much disk space as possible in the memory that MySQL can't use directly? It depends, if your datafile is less then 16 GB then the system cache can help, but fill up the innodb_buffer_pool you'll get better performance. Think of innodb as being its own virtual filesystem. If you have 16GB it's probably a 64 bit OS, and mysql is available in 64 bit. I think that the problem is that it's *not* a 64 bit OS. It's just an Intel 32bit box with 4GB of memory. And sine MySQL doesn't do PAE, it'll never see that extra memory. Jeremy -- Jeremy D. Zawodny | Perl, Web, MySQL, Linux Magazine, Yahoo! [EMAIL PROTECTED] | http://jeremy.zawodny.com/ [book] High Performance MySQL -- http://highperformancemysql.com/ -- MySQL General Mailing List For list archives: http://lists.mysql.com/mysql To unsubscribe:http://lists.mysql.com/[EMAIL PROTECTED]
RE: InnoDB filesystem
-Original Message- From: Jeremy Zawodny [mailto:[EMAIL PROTECTED] Sent: Thursday, May 13, 2004 4:03 PM To: Dathan Vance Pattishall Cc: 'Tim Cutts'; 'MySQL List' Subject: Re: InnoDB filesystem On Thu, May 13, 2004 at 04:51:27PM -0700, Dathan Vance Pattishall wrote: I think that the problem is that it's *not* a 64 bit OS. It's just an Intel 32bit box with 4GB of memory. And sine MySQL doesn't do PAE, it'll never see that extra memory. Intel box with 4GB? It is possible with a patch like hugemem in Linux but 4GB should only be used 2^32 = 4GB. - The hugemem patch for instance allows you to use all 16 GB but at a performance penalty. Jeremy -- Jeremy D. Zawodny | Perl, Web, MySQL, Linux Magazine, Yahoo! [EMAIL PROTECTED] | http://jeremy.zawodny.com/ [book] High Performance MySQL -- http://highperformancemysql.com/ -- MySQL General Mailing List For list archives: http://lists.mysql.com/mysql To unsubscribe:http://lists.mysql.com/[EMAIL PROTECTED]
Re: InnoDB filesystem
Jeremy Zawodny wrote: On Thu, May 13, 2004 at 04:51:27PM -0700, Dathan Vance Pattishall wrote: -Original Message- From: Tim Cutts [mailto:[EMAIL PROTECTED] Sent: Thursday, May 13, 2004 7:11 AM To: MySQL List Subject: Re: InnoDB filesystem On 13 May 2004, at 3:34 pm, Dan Nelson wrote: Pros: performance and bypassing the filesystem cache. MySQL can't use all that memory itself, so it makes sense to allow the OS to cache as much disk space as possible in the memory that MySQL can't use directly? It depends, if your datafile is less then 16 GB then the system cache can help, but fill up the innodb_buffer_pool you'll get better performance. Think of innodb as being its own virtual filesystem. If you have 16GB it's probably a 64 bit OS, and mysql is available in 64 bit. I think that the problem is that it's *not* a 64 bit OS. It's just an Intel 32bit box with 4GB of memory. And sine MySQL doesn't do PAE, it'll never see that extra memory. Didn't InnoDB gain PAE support on some platforms a little while ago? Best regards, Chris Jeremy -- MySQL General Mailing List For list archives: http://lists.mysql.com/mysql To unsubscribe:http://lists.mysql.com/[EMAIL PROTECTED]
