RE: 64-Bit and INNODB
Ahh, so Linux on 64-bit right now with INNODB is really not much different than on 32-bit x86? XEON CPUs have AWE/PAE which lets them address a 36-bit memory address space, getting past the 4GB addressable limit. 64-bit CPUs obviously can address _much_ more memory in a single chunk. MySQL/INNODB though is still going to be limited to that same 2GB buffer size? Is that correct? Hmmm... We've talked about sponsorship of Innobase to implement PAE on XEON/x86 Linux but making it work on Opteron I think would be more appropriate. Do you want to publicly talk about costs of that implementation Heikki? How many folks here would want this and be willing to pass the hat to make it happen? A low-end 1U Opteron server including RAM and CPU are only marginally more expensive than a (good) low-end similarly-equippped XEON server. In my opinion there is no doubt that it will take off in a big way! PS. RedHat kernels definitely support PAE and that's possibly via an additional patch beyond the stock kernel? Not just in Advanced Server either, this is with the bigmem kernel on a box (standard RedHat 7.3) with 5GB RAM for instance [too bad I can only allocate a bit under 2GB for the INNODB buffer though :-) ] 10:07pm up 1 day, 1:07, 1 user, load average: 0.05, 0.07, 0.08 102 processes: 101 sleeping, 1 running, 0 zombie, 0 stopped CPU0 states: 0.3% user, 8.2% system, 0.0% nice, 90.4% idle CPU1 states: 1.0% user, 0.1% system, 0.0% nice, 98.3% idle Mem: 5318292K av, 4571076K used, 747216K free, 0K shrd, 261444K buff Swap: 2096220K av, 123060K used, 1973160K free 3284376K cached Kernel 2.6.0 definitely makes mention of PAE and support for large amounts of memory. -Original Message- From: Heikki Tuuri [mailto:[EMAIL PROTECTED] Sent: Tuesday, August 26, 2003 4:08 PM To: [EMAIL PROTECTED] Subject: Re: 64-Bit and INNODB Hi! - Original Message - From: Marc Slemko [EMAIL PROTECTED] Newsgroups: mailing.database.mysql Sent: Tuesday, August 26, 2003 6:56 AM Subject: RE: 64-Bit and INNODB On Mon, 25 Aug 2003, Wendell Dingus wrote: I didn't notice a reply to this when first posted. Surely someone has stuffed a lot of memory into an Opteron or Itanium by now and knows the answer. Is a 64-bit Malloc all that is necessary or does INNODB have to specifically support more memory in some other fashion? Heikki? Thanks in advance! well, interestingly according to the innodb release notes, on windows: MySQL/InnoDB-4.1.0, April 3, 2003 * InnoDB now supports up to 64 GB of buffer pool memory in a Windows 32-bit Intel computer. This is possible because InnoDB can use the AWE extension of Windows to address memory over the 4 GB limit of a 32-bit process. A new startup variable innodb_buffer_pool_awe_mem_mb enables AWE and sets the size of the buffer pool in megabytes. not sure what it would take to make that work on linux, but if all you need is more memory, and the fairly reasonable performance hit is ok, you may be a lot better off just getting an x86 box with 8 dimm slots and loading them up with 1 or 2 gig dimms... then making AWE in mysql work on linux. The cost you pay to go the 64 bit box is pretty hefty. We are waiting to see if 64-bit Linux computers take off. Adding the 32-bit Intel AWE support into InnoDB on Linux would be rather easy if someone wants to sponsor the project. I recall AWE itself can be used with the Red Hat Linux Advanced Server, if I remember the OS name right. Best regards, Heikki Tuuri Innobase Oy http://www.innodb.com Foreign keys, transactions, and row level locking for MySQL InnoDB Hot Backup - a hot backup tool for MySQL -- MySQL General Mailing List For list archives: http://lists.mysql.com/mysql To unsubscribe:http://lists.mysql.com/[EMAIL PROTECTED] -- MySQL General Mailing List For list archives: http://lists.mysql.com/mysql To unsubscribe:http://lists.mysql.com/[EMAIL PROTECTED]
Re: 64-Bit and INNODB
Andi, - Original Message - From: Andi Kleen [EMAIL PROTECTED] To: Heikki Tuuri [EMAIL PROTECTED] Sent: Wednesday, August 27, 2003 1:37 AM Subject: Re: 64-Bit and INNODB Heikki Tuuri [EMAIL PROTECTED] writes: Adding the 32-bit Intel AWE support into InnoDB on Linux would be rather easy if someone wants to sponsor the project. I recall AWE itself can be used with the Red Hat Linux Advanced Server, if I remember the OS name right. It can be used with any 2.4 Linux kernel. All you need to do is to create a big file in tmpfs and mmap64() windows out of it. No fancy acronyms needed. good. Thank you for the information! The solution I was thinking of was attaching areas of shared memory to the mysqld process. -Andi Best regards, Heikki -- MySQL General Mailing List For list archives: http://lists.mysql.com/mysql To unsubscribe:http://lists.mysql.com/[EMAIL PROTECTED]
Re: 64-Bit and INNODB
Wendell, - Original Message - From: Wendell Dingus [EMAIL PROTECTED] Newsgroups: mailing.database.mysql Sent: Wednesday, August 27, 2003 6:35 AM Subject: RE: 64-Bit and INNODB Ahh, so Linux on 64-bit right now with INNODB is really not much different than on 32-bit x86? XEON CPUs have AWE/PAE which lets them address a 36-bit memory address space, getting past the 4GB addressable limit. 64-bit CPUs obviously can address _much_ more memory in a single chunk. MySQL/INNODB though is still going to be limited to that same 2GB buffer size? Is that correct? no. For example, people are running InnoDB on a 64-bit Sparc with 4 GB buffer pools. Hmmm... We've talked about sponsorship of Innobase to implement PAE on XEON/x86 Linux but making it work on Opteron I think would be more appropriate. Do you want to publicly talk about costs of that implementation Heikki? How many folks here would want this and be willing to pass the hat to make it happen? But InnoDB already works on 64-bit systems. Nothing to implement. There is an AMD64 binary downloadable from http://www.mysql.com/downloads/mysql-4.0.html. A low-end 1U Opteron server including RAM and CPU are only marginally more expensive than a (good) low-end similarly-equippped XEON server. In my opinion there is no doubt that it will take off in a big way! PS. RedHat kernels definitely support PAE and that's possibly via an additional patch beyond the stock kernel? Not just in Advanced Server either, this is with the bigmem kernel on a box (standard RedHat 7.3) with 5GB RAM for instance [too bad I can only allocate a bit under 2GB for the INNODB buffer though :-) ] 10:07pm up 1 day, 1:07, 1 user, load average: 0.05, 0.07, 0.08 102 processes: 101 sleeping, 1 running, 0 zombie, 0 stopped CPU0 states: 0.3% user, 8.2% system, 0.0% nice, 90.4% idle CPU1 states: 1.0% user, 0.1% system, 0.0% nice, 98.3% idle Mem: 5318292K av, 4571076K used, 747216K free, 0K shrd, 261444K buff Swap: 2096220K av, 123060K used, 1973160K free 3284376K cached Kernel 2.6.0 definitely makes mention of PAE and support for large amounts of memory. Andi Kleen just wrote that all 2.4.xx kernels support PAE. Best regards, Heikki Tuuri Innobase Oy http://www.innodb.com Foreign keys, transactions, and row level locking for MySQL InnoDB Hot Backup - a hot backup tool for MySQL -Original Message- From: Heikki Tuuri [mailto:[EMAIL PROTECTED] Sent: Tuesday, August 26, 2003 4:08 PM To: [EMAIL PROTECTED] Subject: Re: 64-Bit and INNODB Hi! - Original Message - From: Marc Slemko [EMAIL PROTECTED] Newsgroups: mailing.database.mysql Sent: Tuesday, August 26, 2003 6:56 AM Subject: RE: 64-Bit and INNODB On Mon, 25 Aug 2003, Wendell Dingus wrote: I didn't notice a reply to this when first posted. Surely someone has stuffed a lot of memory into an Opteron or Itanium by now and knows the answer. Is a 64-bit Malloc all that is necessary or does INNODB have to specifically support more memory in some other fashion? Heikki? Thanks in advance! well, interestingly according to the innodb release notes, on windows: MySQL/InnoDB-4.1.0, April 3, 2003 * InnoDB now supports up to 64 GB of buffer pool memory in a Windows 32-bit Intel computer. This is possible because InnoDB can use the AWE extension of Windows to address memory over the 4 GB limit of a 32-bit process. A new startup variable innodb_buffer_pool_awe_mem_mb enables AWE and sets the size of the buffer pool in megabytes. not sure what it would take to make that work on linux, but if all you need is more memory, and the fairly reasonable performance hit is ok, you may be a lot better off just getting an x86 box with 8 dimm slots and loading them up with 1 or 2 gig dimms... then making AWE in mysql work on linux. The cost you pay to go the 64 bit box is pretty hefty. We are waiting to see if 64-bit Linux computers take off. Adding the 32-bit Intel AWE support into InnoDB on Linux would be rather easy if someone wants to sponsor the project. I recall AWE itself can be used with the Red Hat Linux Advanced Server, if I remember the OS name right. Best regards, Heikki Tuuri Innobase Oy http://www.innodb.com Foreign keys, transactions, and row level locking for MySQL InnoDB Hot Backup - a hot backup tool for MySQL -- MySQL General Mailing List For list archives: http://lists.mysql.com/mysql To unsubscribe: http://lists.mysql.com/[EMAIL PROTECTED] -- MySQL General Mailing List For list archives: http://lists.mysql.com/mysql To unsubscribe: http://lists.mysql.com/[EMAIL PROTECTED] -- MySQL General Mailing List For list archives: http://lists.mysql.com/mysql To unsubscribe:http://lists.mysql.com/[EMAIL PROTECTED]
RE: 64-Bit and INNODB
On Mon, 25 Aug 2003, Wendell Dingus wrote: I didn't notice a reply to this when first posted. Surely someone has stuffed a lot of memory into an Opteron or Itanium by now and knows the answer. Is a 64-bit Malloc all that is necessary or does INNODB have to specifically support more memory in some other fashion? Heikki? Thanks in advance! well, interestingly according to the innodb release notes, on windows: MySQL/InnoDB-4.1.0, April 3, 2003 * InnoDB now supports up to 64 GB of buffer pool memory in a Windows 32-bit Intel computer. This is possible because InnoDB can use the AWE extension of Windows to address memory over the 4 GB limit of a 32-bit process. A new startup variable innodb_buffer_pool_awe_mem_mb enables AWE and sets the size of the buffer pool in megabytes. not sure what it would take to make that work on linux, but if all you need is more memory, and the fairly reasonable performance hit is ok, you may be a lot better off just getting an x86 box with 8 dimm slots and loading them up with 1 or 2 gig dimms... then making AWE in mysql work on linux. The cost you pay to go the 64 bit box is pretty hefty. -- MySQL General Mailing List For list archives: http://lists.mysql.com/mysql To unsubscribe:http://lists.mysql.com/[EMAIL PROTECTED]
Re: 64-Bit and INNODB
Hi! - Original Message - From: Marc Slemko [EMAIL PROTECTED] Newsgroups: mailing.database.mysql Sent: Tuesday, August 26, 2003 6:56 AM Subject: RE: 64-Bit and INNODB On Mon, 25 Aug 2003, Wendell Dingus wrote: I didn't notice a reply to this when first posted. Surely someone has stuffed a lot of memory into an Opteron or Itanium by now and knows the answer. Is a 64-bit Malloc all that is necessary or does INNODB have to specifically support more memory in some other fashion? Heikki? Thanks in advance! well, interestingly according to the innodb release notes, on windows: MySQL/InnoDB-4.1.0, April 3, 2003 * InnoDB now supports up to 64 GB of buffer pool memory in a Windows 32-bit Intel computer. This is possible because InnoDB can use the AWE extension of Windows to address memory over the 4 GB limit of a 32-bit process. A new startup variable innodb_buffer_pool_awe_mem_mb enables AWE and sets the size of the buffer pool in megabytes. not sure what it would take to make that work on linux, but if all you need is more memory, and the fairly reasonable performance hit is ok, you may be a lot better off just getting an x86 box with 8 dimm slots and loading them up with 1 or 2 gig dimms... then making AWE in mysql work on linux. The cost you pay to go the 64 bit box is pretty hefty. We are waiting to see if 64-bit Linux computers take off. Adding the 32-bit Intel AWE support into InnoDB on Linux would be rather easy if someone wants to sponsor the project. I recall AWE itself can be used with the Red Hat Linux Advanced Server, if I remember the OS name right. Best regards, Heikki Tuuri Innobase Oy http://www.innodb.com Foreign keys, transactions, and row level locking for MySQL InnoDB Hot Backup - a hot backup tool for MySQL -- MySQL General Mailing List For list archives: http://lists.mysql.com/mysql To unsubscribe:http://lists.mysql.com/[EMAIL PROTECTED]
