Re: 32-bit vs AMD64 on Opteron for LAMP server
Jim Crilly wrote: On 07/06/07 10:40:11PM -0500, Neil Gunton wrote: Adam Stiles wrote: You won't be able to use all of your 4GB RAM with a 32-bit kernel. A 32-bit processor only has 4GB of addressing space, and that has to be shared between memory and peripherals. Really? I thought that the only limitation was on individual processes not having more than 4GB available, but the entire system as a whole could address a lot more than that. But I could be wrong. For 32-bit systems only if the kernel is compiled with CONFIG_HIGHMEM64G enabled so you need one of the bigmem kernels. And the BIOS on the machine has to support remapping the lost memory above the 4G mark, if it won't do that for you there's nothing you can do to get access to that memory. The per-process limit will be 3G since 4G is the max addressable and 1G of that space is reserved for the kernel. And part of the 3G will be used for the binary itself, shared libraries, mmap'd files, etc so you'll never even get the full 3G out of a single process. Ok, here's another thought: It's easy to get into a purist frame of mind with this where you end up obsessing over numbers like 3 GB or 4 GB or 64 GB. But taking a step back, does any real world process, especially in the LAMP stack, actually need that amount of memory? I know MySQL can probably use whatever you want to throw at it, for index buffer caching particularly, but I can only envisage giving MySQL 3 GB if the total system RAM was a lot more than that - probably at least 6 GB, and I don't plan on doing that soon with this server. I'm not sure if this was mentioned but another option would be to install the 32-bit i386 distribution but run a 64-bit kernel, that way each 32-bit process would have 4G of VM since the kernel wouldn't have to share their address space and you would also have the option of running some select 64-bit binaries if you find that something needs more VM. I didn't even know you could do that! Sounds interesting, though. Seems a little weird mixing the two modes, because I would have thought that processes talking to the kernel would need to be using the same underlying libs and agree on what size an integer is, for example. But evidently it's possible, so thanks for the idea. /Neil -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: 32-bit vs AMD64 on Opteron for LAMP server
On 07/07/07 07:36:26PM -0400, Roberto C. Sánchez wrote: On Sat, Jul 07, 2007 at 05:39:54PM -0400, Jim Crilly wrote: On 07/07/07 04:45:57PM -0400, Roberto C. Sánchez wrote: The stock Debian kernels are configured like this: CONFIG_HIGHMEM4G=y # CONFIG_HIGHMEM64G is not set CONFIG_HIGHMEM=y So, if you have a machine with 4-64 GB RAM, then a custom kernel is in order. Of course, as far as the BIOS goes, if the machine supports more than 4 GB RAM, then the BIOS should as well. After all, why would someone manufacture a machine that can handle more than 4 GB RAM and then put in a BIOS that cannot? No, even with just 4G you need CONFIG_HIGHMEM64G because to access the memory from ~3.5G-4G you need to remap it above the 4G mark since those addresses were stolen by the various hardware components in your system so you need a kernel able to address 4G. Please read again my first sentence. You and I are in agreement on this, just saying it in different ways. Whoops, yea I did misread that. But a custom kernel shouldn't be necessary because on 32-bit systems the 686-bigmem kernel has CONFIG_HIGHMEM64G and on 64-bit systems there's nothing special needed. Regards, -Roberto Jim. -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: 32-bit vs AMD64 on Opteron for LAMP server
On 07/08/07 09:49:32AM -0500, Neil Gunton wrote: Jim Crilly wrote: For 32-bit systems only if the kernel is compiled with CONFIG_HIGHMEM64G enabled so you need one of the bigmem kernels. And the BIOS on the machine has to support remapping the lost memory above the 4G mark, if it won't do that for you there's nothing you can do to get access to that memory. The per-process limit will be 3G since 4G is the max addressable and 1G of that space is reserved for the kernel. And part of the 3G will be used for the binary itself, shared libraries, mmap'd files, etc so you'll never even get the full 3G out of a single process. Ok, here's another thought: It's easy to get into a purist frame of mind with this where you end up obsessing over numbers like 3 GB or 4 GB or 64 GB. But taking a step back, does any real world process, especially in the LAMP stack, actually need that amount of memory? I know MySQL can probably use whatever you want to throw at it, for index buffer caching particularly, but I can only envisage giving MySQL 3 GB if the total system RAM was a lot more than that - probably at least 6 GB, and I don't plan on doing that soon with this server. Well I can't really comment on how useful it would be because that would be dependent on the usage patterns of the installation and I've never had to deal with a LAMP installation of any real size. But the memory limits are virtual, not physical. So if you had planned on giving MySQL 3G for anything you would need a larger VM space because a chunk of it will already be used. I'm not sure if this was mentioned but another option would be to install the 32-bit i386 distribution but run a 64-bit kernel, that way each 32-bit process would have 4G of VM since the kernel wouldn't have to share their address space and you would also have the option of running some select 64-bit binaries if you find that something needs more VM. I didn't even know you could do that! Sounds interesting, though. Seems a little weird mixing the two modes, because I would have thought that processes talking to the kernel would need to be using the same underlying libs and agree on what size an integer is, for example. But evidently it's possible, so thanks for the idea. It's essentially the same concept as installing the 64-bit release and then putting a 32-bit chroot on that box to run things like flash, the same kernel is executing both sets of executables. Most processes talk to the kernel very little and do so in via well defined syscalls so there's no problem running 32-bit binaries on a 64-bit kernel. Occasionally something that does do odd things with the kernel, like the iptables userland stuff, can have issues but those are special cases. The kernel doesn't use any libraries itself so that's not a problem, but yes you do need all the right versions and builds of any dependencies. So if you have a 32-bit userland and decide that you want to run a 64-bit MySQL you need to build 64-bit versions of MySQL and all of the libraries that it needs before it'll run. I'm not sure if you could setup a 64-bit chroot on the 32-bit system to make that simpler but if so that would take away the pain of having to build anything. /Neil Jim. -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: 32-bit vs AMD64 on Opteron for LAMP server
Hi, If I understand right you need to rebuild all the system because you need to repartition, If you need the software in sarge why not have sarge and use your already patched kernel? Try to patch the etch kernel before you go and even you could spend a quickly down time on the server previous to rebuild in situ just patch the kernel install reboot and ask for another reboot if the things don't go... This is second thread about this server rebuild if my memory is accurate... From my point of view you need sarge and you already have experience with, don't reinstall everything just move to the right place, backup data delete and recreate part of the new partitions move the base system, delete and recreate again the rest of the partitions and place in the final place the base system. There is a possibility that you could upgrade everything to etch and preserve your kernel but I have doubts in that point, certainly in this list could answer it. On the othe hand , are your sure about all the system could operate without problems with etch i386? If you are sure don't waste your time and make your life easy installing it, if that is 10% less powerfull in the other way that is 100% more maintainable. On 7/6/07, Neil Gunton [EMAIL PROTECTED] wrote: Roberto C. Sánchez wrote: On Fri, Jul 06, 2007 at 04:07:23PM +0100, Adam Stiles wrote: You won't be able to use all of your 4GB RAM with a 32-bit kernel. A 32-bit processor only has 4GB of addressing space, and that has to be shared between memory and peripherals. Not true. With PAE, a 36-bit address is available, allowing access to 64 GB of RAM. What does not change, however, is that with a 32-bit kernel no single process can address more than 4 GB of RAM. That sounds very much like what I have read as well. You'd also do better with Apache 2.0 or 2.2, as long as you use the prefork version (which is more compatible with PHP, if that's your chosen P). The breaking-up of the configuration files is a bit of a pain to deal with, but worth it in the long run (I knocked up a Perl script to break up a 1.3-style configuration file into 2.0-style snippets; e-mail me if you are interested, on-list if you think others would be interested). Otherwise it's just like 1.3, only faster. This is true. Any pain spent transitioning from Apache v1 to Apache v2 or 2.2 is well worth it. You're probably right; however, as is probably usual with production environments, I never really have the time or inclination to take time out to do such a major shift. Apache 1.3 works just fine for me, I really doubt there is that much difference between using apache 1.3 and apache 2, since the limiting factor for me is almost certainly not in the number of raw static connections that can be served, but rather the mod_perl backend communicating with the mysql database server. That is the heavyweight here, not apache itself. CPU used to do complex sql queries, and disk I/O in serving those queries, is what is going to kill performance, not how fast apache can fork a new process. Thanks, /Neil -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED] -- Perhaps the depth of love can be calibrated by the number of different selves that are actively involved in a given relationship. Carl Sagan (Contact) Jaime Ochoa Malagón Integrated Technology Tel: (55) 52 54 26 10
Re: 32-bit vs AMD64 on Opteron for LAMP server
On Fri, Jul 06, 2007 at 10:24:14PM -0500, Neil Gunton wrote: Fairly early in the install, there's a menu item continue install from ssh. Is there no way to get remote access to the console so that the datacenter people only have to put CD-bin1 into the drive? I have never seen this option (I recently installed Etch from scratch on my home workstation, after my hard drive died). How is this accessed? Where do you get the (no doubt arcane) information that is needed to access the option, since it plainly wasn't visible during the stock install? Read the installation manual (and the release notes). I also chose expert mode (not -GUI), curses interface. You can do software raid right from the installer. My box has two drives, with LVM over raid1 for the system including swap. Grub can boot off of raid1 so put /boot on raid1 (no LVM) partition. Again, I saw no option for software RAID. I did see an option for LVM during the partitioning section, but nothing at all about RAID. How would I enable that? When you select a drive or partition, there's the use-as line. Instead of selecting a filesystem type, choose the line that refers to RAID. Once you have partitions marked for RAID, a new menu item in the partitioner appears; something to the effect of set up raid. Once you have your raid set up, then mark the md*s for use as physical volume for LVM, then you go through that and create your LVs. Then you select your LVs and use as your filesystems. Lable each filesystem so that after the install you can use LABEL= in your kernel command line and in fstab to avoid the device renaming issue. -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: 32-bit vs AMD64 on Opteron for LAMP server
Jaime Ochoa Malagón wrote: If I understand right you need to rebuild all the system because you need to repartition, Right If you need the software in sarge why not have sarge and use your already patched kernel? ??? I never said I need software in Sarge. I would much rather use Etch, because it is more up to date. This is second thread about this server rebuild if my memory is accurate... Yes, I have been planning this for a while. Sorry if it seems repetitive. On the othe hand , are your sure about all the system could operate without problems with etch i386? Yes, I have my workstation at home using Etch for some time now, and it has a development environment which includes everything needed to run the website. I would like to run Etch simply because if I am re-installing anyway, then might as well upgrade to current in the process. /Neil -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: 32-bit vs AMD64 on Opteron for LAMP server
On 07/06/07 10:40:11PM -0500, Neil Gunton wrote: Adam Stiles wrote: You won't be able to use all of your 4GB RAM with a 32-bit kernel. A 32-bit processor only has 4GB of addressing space, and that has to be shared between memory and peripherals. Really? I thought that the only limitation was on individual processes not having more than 4GB available, but the entire system as a whole could address a lot more than that. But I could be wrong. For 32-bit systems only if the kernel is compiled with CONFIG_HIGHMEM64G enabled so you need one of the bigmem kernels. And the BIOS on the machine has to support remapping the lost memory above the 4G mark, if it won't do that for you there's nothing you can do to get access to that memory. The per-process limit will be 3G since 4G is the max addressable and 1G of that space is reserved for the kernel. And part of the 3G will be used for the binary itself, shared libraries, mmap'd files, etc so you'll never even get the full 3G out of a single process. I'm not sure if this was mentioned but another option would be to install the 32-bit i386 distribution but run a 64-bit kernel, that way each 32-bit process would have 4G of VM since the kernel wouldn't have to share their address space and you would also have the option of running some select 64-bit binaries if you find that something needs more VM. The only bad thing about that is you would have to compile whatever 64-bit stuff you need on your own since the only 64-bit packages in i386 seem to be the kernel and a handful of libraries. Thanks again, /Neil Jim. -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: 32-bit vs AMD64 on Opteron for LAMP server
On Sat, Jul 07, 2007 at 04:40:00PM -0400, Jim Crilly wrote: For 32-bit systems only if the kernel is compiled with CONFIG_HIGHMEM64G enabled so you need one of the bigmem kernels. And the BIOS on the machine has to support remapping the lost memory above the 4G mark, if it won't do that for you there's nothing you can do to get access to that memory. The stock Debian kernels are configured like this: CONFIG_HIGHMEM4G=y # CONFIG_HIGHMEM64G is not set CONFIG_HIGHMEM=y So, if you have a machine with 4-64 GB RAM, then a custom kernel is in order. Of course, as far as the BIOS goes, if the machine supports more than 4 GB RAM, then the BIOS should as well. After all, why would someone manufacture a machine that can handle more than 4 GB RAM and then put in a BIOS that cannot? Regards, -Roberto -- Roberto C. Sánchez http://people.connexer.com/~roberto http://www.connexer.com signature.asc Description: Digital signature
Re: 32-bit vs AMD64 on Opteron for LAMP server
On 07/07/07 04:45:57PM -0400, Roberto C. Sánchez wrote: On Sat, Jul 07, 2007 at 04:40:00PM -0400, Jim Crilly wrote: For 32-bit systems only if the kernel is compiled with CONFIG_HIGHMEM64G enabled so you need one of the bigmem kernels. And the BIOS on the machine has to support remapping the lost memory above the 4G mark, if it won't do that for you there's nothing you can do to get access to that memory. The stock Debian kernels are configured like this: CONFIG_HIGHMEM4G=y # CONFIG_HIGHMEM64G is not set CONFIG_HIGHMEM=y So, if you have a machine with 4-64 GB RAM, then a custom kernel is in order. Of course, as far as the BIOS goes, if the machine supports more than 4 GB RAM, then the BIOS should as well. After all, why would someone manufacture a machine that can handle more than 4 GB RAM and then put in a BIOS that cannot? No, even with just 4G you need CONFIG_HIGHMEM64G because to access the memory from ~3.5G-4G you need to remap it above the 4G mark since those addresses were stolen by the various hardware components in your system so you need a kernel able to address 4G. Regards, -Roberto Jim. -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: 32-bit vs AMD64 on Opteron for LAMP server
Hi! Neil Gunton [EMAIL PROTECTED] írta 2007-07-06 22:24-kor: Unfortunately, the RAID card I have doesn't seem to have been a type that really took off in any significant way. I don't know of anyone else who has one. If anyone does happen to have a spare Adaptec Smart RAID 2015S lying around in an unused server, then I'd love to hear whether they can try installing Etch and patching the kernel with the new dpt_i2o. But somehow I think anyone with such a server is probably using it. I have Installed and do maintain a server which has Adaptec 2410S. from lspci -v 's output: 01:02.0 RAID bus controller: Adaptec AAC-RAID (rev 01) Subsystem: Adaptec AAR-2410SA PCI SATA 4ch (Jaguar II) Flags: bus master, 66MHz, slow devsel, latency 64, IRQ 233 Memory at dc00 (32-bit, prefetchable) [size=64M] Expansion ROM at fea0 [disabled] [size=32K] Capabilities: [80] Power Management version 2 I don't like it. It works, anyway. I had never experienced any problem with it, since I installed, but I like 3ware cards better, since they have utils with it's driver CD, which is capable to interact with its /dev/twaX controlling device. I can see the rebuilding status online, I can start rebuild, build hw raid online, without shutting the OS down and need to get into the bios. tw_cli is my best friend in this topic ;-) But for this adaptec sh17, I cannot found any controlling utility. Adaptec's site seemed kind of chaos to me, and in case if you want to download sg. from them U need to register through dozens of forms, and you can double it, if U don't live in US. (So, maybe they had driver, but I had have no patience.) Anyway, It works, with etch's builtin aacraid driver, it shows the volume name I had set up in its bios, etc: Adaptec aacraid driver (1.1-5[2409]-mh2) ... AAC0: kernel 4.2-0[8205] AAC0: monitor 4.2-0[8205] AAC0: bios 4.2-0[8205] AAC0: serial 3b2a5 scsi0 : aacraid Vendor: Adaptec Model: zafir Rev: V1.0 Type: Direct-Access ANSI SCSI revision: 02 ... libata version 2.00 loaded. SCSI device sda: 1874989056 512-byte hdwr sectors (959994 MB) sda: assuming Write Enabled sda: assuming drive cache: write through Sincerely, PÁSZTOR György -- ---[ Free Software ISOs - http://www.fsn.hu/?f=download ]--- -- PÁSZTOR György e-mail: [EMAIL PROTECTED] Free Software Network (FSN.HU) cell.: +3620 512 3335
Re: 32-bit vs AMD64 on Opteron for LAMP server
On Sat, Jul 07, 2007 at 05:39:54PM -0400, Jim Crilly wrote: On 07/07/07 04:45:57PM -0400, Roberto C. Sánchez wrote: The stock Debian kernels are configured like this: CONFIG_HIGHMEM4G=y # CONFIG_HIGHMEM64G is not set CONFIG_HIGHMEM=y So, if you have a machine with 4-64 GB RAM, then a custom kernel is in order. Of course, as far as the BIOS goes, if the machine supports more than 4 GB RAM, then the BIOS should as well. After all, why would someone manufacture a machine that can handle more than 4 GB RAM and then put in a BIOS that cannot? No, even with just 4G you need CONFIG_HIGHMEM64G because to access the memory from ~3.5G-4G you need to remap it above the 4G mark since those addresses were stolen by the various hardware components in your system so you need a kernel able to address 4G. Please read again my first sentence. You and I are in agreement on this, just saying it in different ways. Regards, -Roberto -- Roberto C. Sánchez http://people.connexer.com/~roberto http://www.connexer.com signature.asc Description: Digital signature
Re: 32-bit vs AMD64 on Opteron for LAMP server
On 7/8/07, PÁSZTOR György [EMAIL PROTECTED] wrote: Hi! Neil Gunton [EMAIL PROTECTED] írta 2007-07-06 22:24-kor: Unfortunately, the RAID card I have doesn't seem to have been a type that really took off in any significant way. I don't know of anyone else who has one. If anyone does happen to have a spare Adaptec Smart RAID 2015S lying around in an unused server, then I'd love to hear whether they can try installing Etch and patching the kernel with the new dpt_i2o. But somehow I think anyone with such a server is probably using it. I have Installed and do maintain a server which has Adaptec 2410S. from lspci -v 's output: 01:02.0 RAID bus controller: Adaptec AAC-RAID (rev 01) Subsystem: Adaptec AAR-2410SA PCI SATA 4ch (Jaguar II) Flags: bus master, 66MHz, slow devsel, latency 64, IRQ 233 Memory at dc00 (32-bit, prefetchable) [size=64M] Expansion ROM at fea0 [disabled] [size=32K] Capabilities: [80] Power Management version 2 I don't like it. It works, anyway. I had never experienced any problem with it, since I installed, but I like 3ware cards better, since they have utils with it's driver CD, which is capable to interact with its /dev/twaX controlling device. We have servers with adaptec hardware raid 2000, 2010, 2015 and 2020 cards. We had some problems with broken cards and a crashing RAID array. Performance seems quite good and management can be done with the arcconf utility. We use the dpt_i2o (and for the 2020 aacraid) driver everywhere since I have seen the i2o block driver eating my data once:) I can see the rebuilding status online, I can start rebuild, build hw raid online, without shutting the OS down and need to get into the bios. tw_cli is my best friend in this topic ;-) But for this adaptec sh17, I cannot found any controlling utility. Adaptec's site seemed kind of chaos to me, and in case if you want to download sg. from them U need to register through dozens of forms, and you can double it, if U don't live in US. (So, maybe they had driver, but I had have no patience.) Try downloading the Storage manager and check out the 'arcconf' utility. Geert
Re: 32-bit vs AMD64 on Opteron for LAMP server
[EMAIL PROTECTED] wrote: I don't have any facts or numbers to bring forward, but you probably won't notice that much of a difference. What do you base this opinion on? Having said that, I would say go for the AMD64 port. I don't think the install is anywhere near less straightforward than the 32bit port. The AMD64 port is less straightforward because they do not include the dpt_i2o driver, which is necessary for my RAID card. Instead, I have to either use the i2o_block driver, which I gather has had some reliability issues under load, or else build a patched kernel with dpt_i2o, which isn't ideal since it always complicates and lengthens the install. Also, for some reason, my own kernels always seem to end up being slower than the stock kernels, even though I am careful to go through and build in/enable everything very specific to the hardware. But since you are running 4 Gig of RAM, take full advantage of it by running amd64. My understanding is that 32-bit can utilize up to 4 GB as well, it's when you have more than 4GB that it starts to matter. The developers have done a great job getting this port official and you won't be disappointed with it. Its rock solid. I know it's solid, it's been running in my server for over two years. I'm wondering about real world speed difference between 32-bit and 64-bit given the fact that the 64-bit world is still not as complete in terms of software support (dpt_i2o being case in point, but there is, I assume, other software that isn't 64-bit safe yet as well). The 32-bit version is also rock solid, even more so probably. If you can, try both in a test environment, and then make your decision. Perhaps there are some apache, or mysql benchmarks you can try? This is my only server. It's production, and in colo. I don't have money to have a multi-server setup at this time. When I take it down, the whole shebang is down, and I need to get things back up again asap. So it's going to be, ideally, a process of backing up the essential data, rebuild, restore essential data, reconfigure for Etch, and go. Also, the server is in Chicago IL, I am in Saint Louis MO, I don't have much time up there. The colo company has to accompany me in to the datacenter for security. They wanted to charge me $100 per hour to have someone be there with me, but they are doing me a favor by not charging me for their time. I know, it seems silly, but that's how they do it. I am getting a relatively good deal on the bandwidth so it's worth keeping them happy. Upshot is, I don't have unlimited time/opportunity to be at the console installing and reinstalling operating systems, or mucking around for that matter trying to get kernels patched. I know that once we have the base system up I can do stuff via ssh, but we're talking about console time here. Hence my desire to see how 32-bit would compare performance-wise, because I know that has dpt_i2o built-in and will work right out of the box. Oh, and I would ask you if you have considered Software raid instead of using an adaptec raid card? If you have time, try benchmarking between them. You might be suprised to see which one actually works better for you. I don't know why it would be faster to use up my main CPU cycles doing RAID rather than using the hardware RAID that is already in the box. The guy that built my server seemed to think that this RAID solution was very appropriate for this setup. In any case, see my previous paragraph - I won't have time to mess around with testing different setups. Also, from what I have seen of software RAID in linux, you still have to mess around with making a new kernel with the RAID personalities built into the kernel (rather than modules) to be able to boot off a software RAID partition. The Adaptec makes it all transparent to me, the 4 drives appear as one single drive and RAID 10 is done under the covers. Does anybody else have any opinions on whether software RAID would be faster than using the built-in Adaptec SmartRaid 2015S card? For reference, my travails using dpt_i2o with AMD64 were documented here: http://lists.debian.org/debian-amd64/2005/09/msg00201.html Thanks, /Neil -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: 32-bit vs AMD64 on Opteron for LAMP server
Quoting Neil Gunton [EMAIL PROTECTED]: [EMAIL PROTECTED] wrote: I don't have any facts or numbers to bring forward, but you probably won't notice that much of a difference. What do you base this opinion on? Nothing really, I've setup a handful of amd64 servers and a whack of i386 ones. Alot of them web servers running basic php and mysql stuff. Speeds felt the same. Having said that, I would say go for the AMD64 port. I don't think the install is anywhere near less straightforward than the 32bit port. The AMD64 port is less straightforward because they do not include the dpt_i2o driver, which is necessary for my RAID card. Instead, I have to either use the i2o_block driver, which I gather has had some reliability issues under load, or else build a patched kernel with dpt_i2o, which isn't ideal since it always complicates and lengthens the install. Also, for some reason, my own kernels always seem to end up being slower than the stock kernels, even though I am careful to go through and build in/enable everything very specific to the hardware. Ah, yes. If your missing some hardware drivers, it can be a pain. But since you are running 4 Gig of RAM, take full advantage of it by running amd64. My understanding is that 32-bit can utilize up to 4 GB as well, it's when you have more than 4GB that it starts to matter. I always thought that 32 bit sarge with 4G ram, would only show about 3.3G of available RAM. I could be wrong, and perhaps this doesn't happen in etch. Oh, and I would ask you if you have considered Software raid instead of using an adaptec raid card? If you have time, try benchmarking between them. You might be suprised to see which one actually works better for you. I don't know why it would be faster to use up my main CPU cycles doing RAID rather than using the hardware RAID that is already in the box. The guy that built my server seemed to think that this RAID solution was very appropriate for this setup. In any case, see my previous paragraph - I won't have time to mess around with testing different setups. Also, from what I have seen of software RAID in linux, you still have to mess around with making a new kernel with the RAID personalities built into the kernel (rather than modules) to be able to boot off a software RAID partition. The Adaptec makes it all transparent to me, the 4 drives appear as one single drive and RAID 10 is done under the covers. Does anybody else have any opinions on whether software RAID would be faster than using the built-in Adaptec SmartRaid 2015S card? Etch has full SW raid support right out of the install. No messing with kernels. You can even install raid on the / during the install if you wanted. I didn't want to get into a hardware vs software raid conversation, but figured if you had a few days of down time, it would nice to run some tests before hand. Cheers, Mike This message was sent using IMP, the Internet Messaging Program.
Re: 32-bit vs AMD64 on Opteron for LAMP server
On Fri, Jul 06, 2007 at 07:29:25AM -0700, [EMAIL PROTECTED] wrote: Quoting Neil Gunton [EMAIL PROTECTED]: Does anybody else have any opinions on whether software RAID would be faster than using the built-in Adaptec SmartRaid 2015S card? Etch has full SW raid support right out of the install. No messing with kernels. You can even install raid on the / during the install if you wanted. I didn't want to get into a hardware vs software raid conversation, but figured if you had a few days of down time, it would nice to run some tests before hand. The determining factor for whether to use hardware RAID or software RAID is whether your hardware RAID actually does hardware RAID. Do a Google search on fake RAID to get an idea of what I mean. Many hardware RAID cards, especially the cheaper varieties, are nothing more than proprietary software RAID implemented in the card BIOS. This is something that you DO NOT want. Use something that has REAL hardware RAID (Intel MegaRAID, HP SmartArray, 3Ware, etc), or just use the Linux built-in software RAID. Regards, -Roberto -- Roberto C. Sánchez http://people.connexer.com/~roberto http://www.connexer.com signature.asc Description: Digital signature
Re: 32-bit vs AMD64 on Opteron for LAMP server
On Thursday 05 July 2007 22:07, Neil Gunton wrote: Hi all, I am going to be rebuilding a server in August which has been running Sarge AMD64 for the last couple of years in a colo environment. I am going to do a total rebuild (need to repartition the disks). Since I am using the dpt_i2o drivers for Adaptec RAID, which I had a hell of a problem getting to work under Sarge AMD64 (and still aren't in the default modules for Etch), I am considering all options, including simply using 32-bit Etch on the new install. This is a LAMP server, running apache 1.3, mysql 5. You won't be able to use all of your 4GB RAM with a 32-bit kernel. A 32-bit processor only has 4GB of addressing space, and that has to be shared between memory and peripherals. You'd also do better with Apache 2.0 or 2.2, as long as you use the prefork version (which is more compatible with PHP, if that's your chosen P). The breaking-up of the configuration files is a bit of a pain to deal with, but worth it in the long run (I knocked up a Perl script to break up a 1.3-style configuration file into 2.0-style snippets; e-mail me if you are interested, on-list if you think others would be interested). Otherwise it's just like 1.3, only faster. If your RAID card is one of the ones that uses a binary-only driver, ignore it and use the open source drivers with md RAID instead -- md is faster than any manufacturer's proprietary alternative (anything that needs a driver is fake RAID. True hardware RAID never needs a special driver; the array just shows up as a single drive). Beside the non-polluted kernel, you also get the advantage that you can have your swap area without redundancy, hence running as fast as possible (just set up the partitions as separate swap areas). -- AJS -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: 32-bit vs AMD64 on Opteron for LAMP server
On July 6, 2007 07:29 am [EMAIL PROTECTED] wrote: Quoting Neil Gunton [EMAIL PROTECTED]: [EMAIL PROTECTED] wrote: I don't have any facts or numbers to bring forward, but you probably won't notice that much of a difference. What do you base this opinion on? Nothing really, I've setup a handful of amd64 servers and a whack of i386 ones. Alot of them web servers running basic php and mysql stuff. Speeds felt the same. Having said that, I would say go for the AMD64 port. I don't think the install is anywhere near less straightforward than the 32bit port. The AMD64 port is less straightforward because they do not include the dpt_i2o driver, which is necessary for my RAID card. Instead, I have to either use the i2o_block driver, which I gather has had some reliability issues under load, or else build a patched kernel with dpt_i2o, which isn't ideal since it always complicates and lengthens the install. Also, for some reason, my own kernels always seem to end up being slower than the stock kernels, even though I am careful to go through and build in/enable everything very specific to the hardware. Ah, yes. If your missing some hardware drivers, it can be a pain. But since you are running 4 Gig of RAM, take full advantage of it by running amd64. My understanding is that 32-bit can utilize up to 4 GB as well, it's when you have more than 4GB that it starts to matter. I always thought that 32 bit sarge with 4G ram, would only show about 3.3G of available RAM. I could be wrong, and perhaps this doesn't happen in etch. Depends on the motherboard. The top 256-768 MB of RAM is reserved for PCI mapped memory space and other devices. Some motherboards and CPUs let you re-map this memory above the 4 GB boundary (making it appear that you have up to 5 GB of RAM) while others don't. AMD systems tend to be better in allowing the remapping. 32-bit systems can use 3 GB without problems. It's when you have more than 3 GB that things start to get weird. 64-bit systems don't have these issues. -- Freddie Cash, LPIC-2 CCNT CCLP Network Support Technician School District 73 (250) 377-HELP [377-4357] [EMAIL PROTECTED] -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: 32-bit vs AMD64 on Opteron for LAMP server
On Fri, Jul 06, 2007 at 04:07:23PM +0100, Adam Stiles wrote: You won't be able to use all of your 4GB RAM with a 32-bit kernel. A 32-bit processor only has 4GB of addressing space, and that has to be shared between memory and peripherals. Not true. With PAE, a 36-bit address is available, allowing access to 64 GB of RAM. What does not change, however, is that with a 32-bit kernel no single process can address more than 4 GB of RAM. You'd also do better with Apache 2.0 or 2.2, as long as you use the prefork version (which is more compatible with PHP, if that's your chosen P). The breaking-up of the configuration files is a bit of a pain to deal with, but worth it in the long run (I knocked up a Perl script to break up a 1.3-style configuration file into 2.0-style snippets; e-mail me if you are interested, on-list if you think others would be interested). Otherwise it's just like 1.3, only faster. This is true. Any pain spent transitioning from Apache v1 to Apache v2 or 2.2 is well worth it. If your RAID card is one of the ones that uses a binary-only driver, ignore it and use the open source drivers with md RAID instead -- md is faster than any manufacturer's proprietary alternative (anything that needs a driver is fake RAID. True hardware RAID never needs a special driver; the array just shows up as a single drive). Beside the non-polluted kernel, you also get the advantage that you can have your swap area without redundancy, hence running as fast as possible (just set up the partitions as separate swap areas). Clearly you don't know what you are talking about here. If you look, there are drivers in the kernel for megaraid (Intel), cciss (HP Smart Array) and 3ware. All of there are certainly hardware RAID. In fact, all of those are also in the kernel and free software drivers, developed primarly by the hardware vendors. In any case, your statement is equivalent to saying A true video card never needs a special driver; the card just shows up as a video device. *Every* device on the system needs a driver of some sort or another. Regards, -Roberto -- Roberto C. Sánchez http://people.connexer.com/~roberto http://www.connexer.com signature.asc Description: Digital signature
Re: 32-bit vs AMD64 on Opteron for LAMP server
On Fri, Jul 06, 2007 at 07:53:18AM -0500, Neil Gunton wrote: [EMAIL PROTECTED] wrote: The AMD64 port is less straightforward because they do not include the dpt_i2o driver, which is necessary for my RAID card. Instead, I have to either use the i2o_block driver, which I gather has had some reliability issues under load, or else build a patched kernel with dpt_i2o, which isn't ideal since it always complicates and lengthens the install. Also, for some reason, my own kernels always seem to end up being slower than the stock kernels, even though I am careful to go through and build in/enable everything very specific to the hardware. Without the driver, will the card just look like a non-raid card or will the drives not be visiable at all? If the drives are visible, forget the hardware raid. Otherwise, would something like module-assistant be able to make your module for you without compiling the whole kernel? But since you are running 4 Gig of RAM, take full advantage of it by running amd64. My understanding is that 32-bit can utilize up to 4 GB as well, it's when you have more than 4GB that it starts to matter. There's also something about the maximum amount of memory that a single process can access in 32-bit but I forget the details. The developers have done a great job getting this port official and you won't be disappointed with it. Its rock solid. I know it's solid, it's been running in my server for over two years. I'm wondering about real world speed difference between 32-bit and 64-bit given the fact that the 64-bit world is still not as complete in terms of software support (dpt_i2o being case in point, but there is, I assume, other software that isn't 64-bit safe yet as well). The 32-bit version is also rock solid, even more so probably. The only software I've found missing is a flash plugin for browsers but there's a fix in testing that I'm told can be dragged into one's Etch box (I'll try it in a bit). If you can, try both in a test environment, and then make your decision. Perhaps there are some apache, or mysql benchmarks you can try? This is my only server. It's production, and in colo. What about a test on any amd64 box. Borrow one for a few hours? I don't have money to have a multi-server setup at this time. When I take it down, the whole shebang is down, and I need to get things back up again asap. So it's going to be, ideally, a process of backing up the essential data, rebuild, restore essential data, reconfigure for Etch, and go. Also, the server is in Chicago IL, I am in Saint Louis MO, I don't have much time up there. The colo company has to accompany me in to the datacenter for security. They wanted to charge me $100 per hour to have someone be there with me, but they are doing me a favor by not charging me for their time. I know, it seems silly, but that's how they do it. I am getting a relatively good deal on the bandwidth so it's worth keeping them happy. Upshot is, I don't have unlimited time/opportunity to be at the console installing and reinstalling operating systems, or mucking around for that matter trying to get kernels patched. I know that once we have the base system up I can do stuff via ssh, but we're talking about console time here. Hence my desire to see how 32-bit would compare performance-wise, because I know that has dpt_i2o built-in and will work right out of the box. Fairly early in the install, there's a menu item continue install from ssh. Is there no way to get remote access to the console so that the datacenter people only have to put CD-bin1 into the drive? I can do an install from scratch on my Athlon64, including the raid setup, in about 20 minutes. The i386 installer looks the same so you could practice what you want to do on a spare i386 where you are and write it out step-by-step so that you minimize down time. Oh, and I would ask you if you have considered Software raid instead of using an adaptec raid card? If you have time, try benchmarking between them. You might be suprised to see which one actually works better for you. I don't know why it would be faster to use up my main CPU cycles doing RAID rather than using the hardware RAID that is already in the box. The guy that built my server seemed to think that this RAID solution was very appropriate for this setup. In any case, see my previous paragraph - I won't have time to mess around with testing different setups. Also, from what I have seen of software RAID in linux, you still have to mess around with making a new kernel with the RAID personalities built into the kernel (rather than modules) to be able to boot off a software RAID partition. The Adaptec makes it all transparent to me, the 4 drives appear as one single drive and RAID 10 is done under the covers. You can do software raid right from the installer. My box has two drives, with LVM over raid1 for the system including swap.
Re: 32-bit vs AMD64 on Opteron for LAMP server
On Fri, Jul 06, 2007 at 04:07:23PM +0100, Adam Stiles wrote: advantage that you can have your swap area without redundancy, hence running as fast as possible (just set up the partitions as separate swap areas). However, with non-raid swap, if something happens to one drive, the system dies due to dead swap. You should never need to swap much anyway so keep it on raid along with the rest of the system. Doug. -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: 32-bit vs AMD64 on Opteron for LAMP server
Douglas Allan Tutty wrote: Without the driver, will the card just look like a non-raid card or will the drives not be visiable at all? If the drives are visible, forget the hardware raid. Without the driver (either dpt_i2o or i2o_block) the hard drives are not visible at all, and you cannot boot. You need the driver to see the drives. As others have pointed out, this doesn't mean the RAID card isn't real hardware RAID; as with any other piece of hardware on the system, the kernel requires a driver to interact with it. The driver is not binary; I have the source code patch from Adaptec, which you need to build into the kernel for AMD64. It is not included with Etch because the community apparently made a decision that two different drivers for the same hardware were not needed, and they decided that they would go with i2o_block. I have heard that i2o_block has some issues with stability, people have talked about corruption and errors under load. Since the i2o_block driver has apparently been abandoned for a while, I don't have a great deal of confidence that these problems were ever really fixed. The dpt_i2o driver has worked very well for me for about two years now, with no problems at all under load, so I would prefer to continue using that. I recently communicated with Mark Salyzyn over at Adaptec, and he forwarded me the current source code version of dpt_i2o which works with the current 2.6 kernels. I haven't had the chance to try it out yet. Even if I go with jbod, I would still need one of these drivers just to see the hard drives. I have no handle on whether the hardware RAID would be better or worse than software RAID. Fairly early in the install, there's a menu item continue install from ssh. Is there no way to get remote access to the console so that the datacenter people only have to put CD-bin1 into the drive? I have never seen this option (I recently installed Etch from scratch on my home workstation, after my hard drive died). How is this accessed? Where do you get the (no doubt arcane) information that is needed to access the option, since it plainly wasn't visible during the stock install? You can do software raid right from the installer. My box has two drives, with LVM over raid1 for the system including swap. Grub can boot off of raid1 so put /boot on raid1 (no LVM) partition. Again, I saw no option for software RAID. I did see an option for LVM during the partitioning section, but nothing at all about RAID. How would I enable that? That's almost 2 years ago. You may find that it just works. If you No, it won't - I know for a fact, from the guy at Adaptec himself, that dpt_i2o is not included with AMD64 because the maintainers decided that they would be going with i2o_block. So I may be able to install using i2o_block, but that isn't the path I want to take since I have no confidence in that module for a production environment, and doing searches on google for it I can find no references to it except from the 2005 timeframe. This does not give me confidence that it was developed or fixed past the time when people were reporting that it had problems. So best case, I can maybe install using i2o_block, and then build my own kernel that uses dpt_i2o. However I need to be at the console to do this, because inevitably there will be hangs and suchlike that require my physical presence at the console. could find someone else with this card, even if they don't usually run debian, they could boot the installer CD and go through the motions and see if the drives appear as one drive (hardware RAID working) or many (no hardware RAID). Unfortunately, the RAID card I have doesn't seem to have been a type that really took off in any significant way. I don't know of anyone else who has one. If anyone does happen to have a spare Adaptec Smart RAID 2015S lying around in an unused server, then I'd love to hear whether they can try installing Etch and patching the kernel with the new dpt_i2o. But somehow I think anyone with such a server is probably using it. Thanks, /Neil -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: 32-bit vs AMD64 on Opteron for LAMP server
[EMAIL PROTECTED] wrote: Etch has full SW raid support right out of the install. No messing with kernels. I didn't see anything about this in the recent home install I did... and I was specifically paying attention to look out for it, since I had just bought two identical 320 GB WD drives to replace the one that had just died. I wanted to build a system that would allow me to have a disk die without having to rebuild everything from scratch again. But I saw nothing in the install process that mentioned an option for software RAID... obviously I missed a beat somewhere! Thanks again, /Neil -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: 32-bit vs AMD64 on Opteron for LAMP server
Adam Stiles wrote: You won't be able to use all of your 4GB RAM with a 32-bit kernel. A 32-bit processor only has 4GB of addressing space, and that has to be shared between memory and peripherals. Really? I thought that the only limitation was on individual processes not having more than 4GB available, but the entire system as a whole could address a lot more than that. But I could be wrong. You'd also do better with Apache 2.0 or 2.2, as long as you use the prefork I am using apache 1.3 because I also use mod_accel and mod_deflate from Igor Sysoev. His modules are 1.3 only, and the proxy module (mod_accel) does some things that mod_proxy didn't seem to provide back when I was trying to build my reverse proxy. The thing that pops up from memory is that mod_accel correctly passed cookies and differentiated requests based on cookies, whereas mod_proxy didn't. So if mod_proxy got two different requests that were for the same URI, but with the only difference being the cookie, then mod_proxy would see them as the same request and (incorrectly, in my book) give the same cached result. I discussed this with some of the httpd developers, and they insisted that differentiating requests based on cookies wasn't in the http spec, and wasn't a proper way to do things; I maintained that since cookies are the universal method of keeping track of sessions and users, thus it makes perfect sense to differentiate requests based on them. But they took a dogmatic line, whereupon Igor popped up and said Hey, my module does what you're needing, and it worked great, and has been for the last four years or so. I have no idea if Apache 2 has progressed to the point of being able to handle cookies correctly, but my discussions did not give me hope back then. Also, it was only recently that mod_perl 2 got to a point where it even worked, let alone was at the same level of reliability that mod_perl 1 was at after years of testing. Finally, Embperl, yet another module essential to my site, wasn't stable in version 2 for a long, long time, and continues to have compatibility issues that break my (large) existing code base. All of this prompted me to write an essay about the patterns I saw, called Rewrites considered harmful, published on my site and on slashdot a few years ago. All in all, I am leery of new systems that claim to get everything right that the old systems didn't, and I like to use well tested systems in production, not new stuff that might be faster for plain vanilla cases, but aren't reliable for more complex situations. Apache 1.3 has worked very well for me, and is rock solid, and for production I don't have time to mess around with completely new APIs that were changed just because some developers needed to feel important that they wrote version 2 of apache. Their little campaign to change the whole apache API caused a huge pain in the ass for many, many people such as myself who had working v1 modules that would have to be debugged and converted to v2 api calls. It's not trivial, and it makes all the old books on writing apache modules out of date. It throws the entire developer community into chaos. Just look at how long it took mod_perl 2 to stabilize. Very stupid situation. So I don't feel particularly well disposed toward apache 2, thanks. If your RAID card is one of the ones that uses a binary-only driver, ignore it and use the open source drivers with md RAID instead -- md is faster than any Not binary, I have the source code patch. manufacturer's proprietary alternative (anything that needs a driver is fake RAID. True hardware RAID never needs a special driver; the array just shows up as a single drive). Beside the non-polluted kernel, you also get the advantage that you can have your swap area without redundancy, hence running as fast as possible (just set up the partitions as separate swap areas). I don't think this is an accurate characterization of what constitutes hardware RAID. Any piece of hardware needs a driver for the kernel to interact with it. Once the Adaptec array has been defined (in its own config at boot time) then it does appear to be a single device. Four hard drives configured as two RAID 1 arrays, then striped into a single RAID 10 device, appears as a single hard drive to the kernel. Thanks again, /Neil -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: 32-bit vs AMD64 on Opteron for LAMP server
Roberto C. Sánchez wrote: On Fri, Jul 06, 2007 at 04:07:23PM +0100, Adam Stiles wrote: You won't be able to use all of your 4GB RAM with a 32-bit kernel. A 32-bit processor only has 4GB of addressing space, and that has to be shared between memory and peripherals. Not true. With PAE, a 36-bit address is available, allowing access to 64 GB of RAM. What does not change, however, is that with a 32-bit kernel no single process can address more than 4 GB of RAM. That sounds very much like what I have read as well. You'd also do better with Apache 2.0 or 2.2, as long as you use the prefork version (which is more compatible with PHP, if that's your chosen P). The breaking-up of the configuration files is a bit of a pain to deal with, but worth it in the long run (I knocked up a Perl script to break up a 1.3-style configuration file into 2.0-style snippets; e-mail me if you are interested, on-list if you think others would be interested). Otherwise it's just like 1.3, only faster. This is true. Any pain spent transitioning from Apache v1 to Apache v2 or 2.2 is well worth it. You're probably right; however, as is probably usual with production environments, I never really have the time or inclination to take time out to do such a major shift. Apache 1.3 works just fine for me, I really doubt there is that much difference between using apache 1.3 and apache 2, since the limiting factor for me is almost certainly not in the number of raw static connections that can be served, but rather the mod_perl backend communicating with the mysql database server. That is the heavyweight here, not apache itself. CPU used to do complex sql queries, and disk I/O in serving those queries, is what is going to kill performance, not how fast apache can fork a new process. Thanks, /Neil -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: 32-bit vs AMD64 on Opteron for LAMP server
Quoting Neil Gunton [EMAIL PROTECTED]: Hi all, I am going to be rebuilding a server in August which has been running Sarge AMD64 for the last couple of years in a colo environment. I am going to do a total rebuild (need to repartition the disks). Since I am using the dpt_i2o drivers for Adaptec RAID, which I had a hell of a problem getting to work under Sarge AMD64 (and still aren't in the default modules for Etch), I am considering all options, including simply using 32-bit Etch on the new install. This is a LAMP server, running apache 1.3, mysql 5. I am aware of some of the benefits of running 64-bit vs 32-bit on an Opteron system, including larger memory addressing, more registers and possibly better performance with larger numbers of connections. However, I am wondering if anyone has actual experience of running a real-world LAMP server, on the same hardware, using the same system, using both 32-bit and 64-bit (not at the same time obviously!). In other words, does anyone have real-world experience of whether AMD64 benefits are really apparent to this kind of application. I currently have 4GB RAM on the server, and no intention to upgrade it soon. I think the disk I/O will become an issue way before memory does (even with kernel disk caching, it's just the impression I get from current experience on my site with upward of 80-100,000 page requests per day and fairly complex sql queries etc). It would certainly make the install much more straightforward if I could just use the i386 version of Etch, but if someone can tell me that there really is substantial difference for LAMP type operation with AMD64, then I'm all ears. I don't have any facts or numbers to bring forward, but you probably won't notice that much of a difference. Having said that, I would say go for the AMD64 port. I don't think the install is anywhere near less straightforward than the 32bit port. But since you are running 4 Gig of RAM, take full advantage of it by running amd64. The developers have done a great job getting this port official and you won't be disappointed with it. Its rock solid. If you can, try both in a test environment, and then make your decision. Perhaps there are some apache, or mysql benchmarks you can try? Oh, and I would ask you if you have considered Software raid instead of using an adaptec raid card? If you have time, try benchmarking between them. You might be suprised to see which one actually works better for you. Cheers, Mike This message was sent using IMP, the Internet Messaging Program.
