[luau] No hard drive, only compact flash card
Aloha, Does anyone have any experience with booting and running an full Linux server install from a 1.0GB CompactFlash Cardor similar. By full server install I mean, apache, sendmail, mysql, gc++, etc. No X needed, only command line. Seems like it would be possible, 500MB for the OS and 50MB for the apps. I wonder how fast/slow they are for access compared to a hard drive. I see 1.0GB card for $299 retail, they will only be getting cheaper/faster/higher density. Aloha, Matt
Re: [luau] No hard drive, only compact flash card
--- Matthew John Darnell [EMAIL PROTECTED] wrote: Aloha, Does anyone have any experience with booting and running an full Linux server install from a 1.0GB CompactFlash Cardor similar. By full server install I mean, apache, sendmail, mysql, gc++, etc. No X needed, only command line. Seems like it would be possible, 500MB for the OS and 50MB for the apps. I wonder how fast/slow they are for access compared to a hard drive. I see 1.0GB card for $299 retail, they will only be getting cheaper/faster/higher density. Seeing as you are taking out what would be the largest and most memory intensive packages in a Linux install (XWin is great, but at a cost), you should be able to do this like any other install. CompactFlash cards are faster on read/write, but the lifespan is only between 100,000-300,000 cycles. If you were to do a system using only flash cards, I would recommend that you place heavy read/write directories, such as /tmp, on a separate card, so that you don't have to reload the server every time that you need to replace the card. Also, from looking around on the 'Net, CF card speeds are slower than ATA/100. Lexar 32X 1GB CR car has a speed of just 4.7MB/s, versus 62 MB/s with ATA/100 drives, after you get through the board. Just my 2 cents, Casey Roberts __ Do you Yahoo!? SBC Yahoo! DSL - Now only $29.95 per month! http://sbc.yahoo.com
Re: [luau] No hard drive, only compact flash card
Matthew John Darnell wrote: Aloha, Does anyone have any experience with booting and running an full Linux server install from a 1.0GB CompactFlash Cardor similar. By full server install I mean, apache, sendmail, mysql, gc++, etc. No X needed, only command line. Seems like it would be possible, 500MB for the OS and 50MB for the apps. I wonder how fast/slow they are for access compared to a hard drive. I see 1.0GB card for $299 retail, they will only be getting cheaper/faster/higher density. Aloha, Matt CF cards are *VERY* slow compared to hard drives, especially on writes. My little 32MB things can manage about 1-4MB/sec reads, but only about 100-300kB/sec writes! This is *REALLY* slow. You will NOT want to even THINK about swapping to it. In other words, make sure you have enough RAM because there won't be any swap. RAM is cheap these days, so this shouldn't be a problem. However, last time I checked, distros like redhat likes to complain a lot if you didn't set up swap for them (I think it used to be that redhat would refuse to install under such a situation?) People tend to overexagerate the erase cycle limitations of flash. CF cards usually do wear patterning to prevent the same sectors from being used over and over, and when they have reached their max usage, that sector is just no longer used and is remapped (like bad sectors on IDE hard drives). The entire card isn't useless. If you're really concerned about this, you can get nicer flash cards that actually present themselves as raw flash, rather than ATA flash, and run a real flash filesystem like jffs2 on it. jffs2 includes on-the-fly compression (which I think can be disabled, but may actually help with read/write speed in this case), and all the bad block handling/wear patterning you could need. However, due to their slowness at writes, I'd reccomend keeping really dynamic things like /tmp in a ramdisk (use tmpfs, it takes up only as much ram as it needs to based on what's in it). You might also want to do something with /var (like unpacking it to a ramdisk at startup, then tarring it up back to CF at shutdown, of course this makes unclean shutdowns REALLY bad). Or, you could just not have logging to /var/log and simply use a ring buffer like is used by busybox's syslog. I'm still curious if even 500MB would be needed for the os. You seem to be used to very bloated desktop oses (like redhat) that are designed to have everything abstracted two or three times (remember, you can always fix the problem by adding another layer of indirection). I will say that I have the os in well under 4MB (where the os is defined as kernel, core apps like stuff in /bin and /sbin, and libraries like glibc in /lib; this does not include /usr of course). Aagin you can save a fair amount on smaller systems by playing tricks with smaller versions of libraries, but on a system with full apps like mysql and gcc, it won't be worth it (as I think gcc completely and utterly requires glibc). Toolchains are big, but they're not that big. I've seen full x86-ARM toolchains in about 50-70MB. But that has to include all the foreign libs. Here, those would be considered part of the os or the apps, depending on their usage, since they are needed to run stuff locally anyway. The static libs will sometimes pose problems because they tend to be rather large, but at least headers are usually pretty small :) --MonMotha
Re: [luau] No hard drive, only compact flash card
- Original Message - From: Casey Roberts [EMAIL PROTECTED] To: [EMAIL PROTECTED] Sent: Sunday, July 06, 2003 10:11 AM Subject: Re: [luau] No hard drive, only compact flash card --- Matthew John Darnell [EMAIL PROTECTED] wrote: Aloha, Does anyone have any experience with booting and running an full Linux server install from a 1.0GB CompactFlash Cardor similar. By full server install I mean, apache, sendmail, mysql, gc++, etc. No X needed, only command line. Seems like it would be possible, 500MB for the OS and 50MB for the apps. I wonder how fast/slow they are for access compared to a hard drive. I see 1.0GB card for $299 retail, they will only be getting cheaper/faster/higher density. Seeing as you are taking out what would be the largest and most memory intensive packages in a Linux install (XWin is great, but at a cost), you should be able to do this like any other install. CompactFlash cards are faster on read/write, but the lifespan is only between 100,000-300,000 cycles. If you were to do a system using only flash cards, I would recommend that you place heavy read/write directories, such as /tmp, on a separate card, so that you don't have to reload the server every time that you need to replace the card. Maybe even a hard drive for the /tmp. If the hard drive crashes the system will go down but it is realativly easy to recover from. Also, from looking around on the 'Net, CF card speeds are slower than ATA/100. Lexar 32X 1GB CR car has a speed of just 4.7MB/s, versus 62 MB/s with ATA/100 drives, after you get through the board. That is a bit slower, I wonder how long a boot up would take. From my very un-scientific observations Linux has less disk thrashing on starup than a Windows box.
Re: [luau] No hard drive, only compact flash card
- Original Message - From: MonMotha [EMAIL PROTECTED] To: [EMAIL PROTECTED] Sent: Sunday, July 06, 2003 1:31 PM Subject: Re: [luau] No hard drive, only compact flash card Matthew John Darnell wrote: Aloha, Does anyone have any experience with booting and running an full Linux server install from a 1.0GB CompactFlash Cardor similar. By full server install I mean, apache, sendmail, mysql, gc++, etc. No X needed, only command line. Seems like it would be possible, 500MB for the OS and 50MB for the apps. I wonder how fast/slow they are for access compared to a hard drive. I see 1.0GB card for $299 retail, they will only be getting cheaper/faster/higher density. Aloha, Matt CF cards are *VERY* slow compared to hard drives, especially on writes. My little 32MB things can manage about 1-4MB/sec reads, but only about 100-300kB/sec writes! This is *REALLY* slow. You will NOT want to even THINK about swapping to it. In other words, make sure you have enough RAM because there won't be any swap. RAM is cheap these days, so this shouldn't be a problem. However, last time I checked, distros like redhat likes to complain a lot if you didn't set up swap for them (I think it used to be that redhat would refuse to install under such a situation?) People tend to overexagerate the erase cycle limitations of flash. CF cards usually do wear patterning to prevent the same sectors from being used over and over, and when they have reached their max usage, that sector is just no longer used and is remapped (like bad sectors on IDE hard drives). The entire card isn't useless. If you're really concerned about this, you can get nicer flash cards that actually present themselves as raw flash, rather than ATA flash, and run a real flash filesystem like jffs2 on it. jffs2 includes on-the-fly compression (which I think can be disabled, but may actually help with read/write speed in this case), and all the bad block handling/wear patterning you could need. I was told once the best minimalist distro was debian. I sure like the functionality of apt-get. An IDE to flash adpater runs about 30 bucks, a lot less than I thought it would. However, due to their slowness at writes, I'd reccomend keeping really dynamic things like /tmp in a ramdisk (use tmpfs, it takes up only as much ram as it needs to based on what's in it). You might also want to do something with /var (like unpacking it to a ramdisk at startup, then tarring it up back to CF at shutdown, of course this makes unclean shutdowns REALLY bad). Or, you could just not have logging to /var/log and simply use a ring buffer like is used by busybox's syslog. I will have to research jffs2 and busybox. I'm still curious if even 500MB would be needed for the os. You seem to be used to very bloated desktop oses (like redhat) that are designed to have everything abstracted two or three times (remember, you can always fix the problem by adding another layer of indirection). I will say that I have the os in well under 4MB (where the os is defined as kernel, core apps like stuff in /bin and /sbin, and libraries like glibc in /lib; this does not include /usr of course). Aagin you can save a fair amount on smaller systems by playing tricks with smaller versions of libraries, but on a system with full apps like mysql and gcc, it won't be worth it (as I think gcc completely and utterly requires glibc). Toolchains are big, but they're not that big. I've seen full x86-ARM toolchains in about 50-70MB. But that has to include all the foreign libs. Here, those would be considered part of the os or the apps, depending on their usage, since they are needed to run stuff locally anyway. The static libs will sometimes pose problems because they tend to be rather large, but at least headers are usually pretty small :) 4MB!?!? with all the important apps? I have seend Linux on a floppy but they were so very limited. I think that is incredible. I will buy one of the converters and try a small install of debian, I already have a 256MB flash card. I am very suprised no one sells PC like this. All of the ones I found were cash registers or the like, no general purpose PC's. I would think this would be great for routers, firewalls, etc. High high availabilty stuff. Thanks, Matt
Re: [luau] No hard drive, only compact flash card
I am very suprised no one sells PC like this. All of the ones I found were cash registers or the like, no general purpose PC's. I would think this would be great for routers, firewalls, etc. High high availabilty stuff. Given all of the limitations that have been described, I don't see how it's all that useful to use CF in a high-availability environment like a router or firewall. PCs are cheap enough now that you can deploy a pair of fully-redundant PCs running Linux (or any other Unix-ish OS), on relatively fast hard drives, for a few hundred dollars. Why would you do anything else? Why bother with the added complexity of CF? -- Mark K. Pettit [EMAIL PROTECTED]
Re: [luau] No hard drive, only compact flash card
There are at least two advantages with regard to flash stick that I can immediately think of relatively to hard drive. First, flash memory has no moving parts. (Is any one aware that the separation between the magnetic head and the disc surface is less than 20 Angstroms and the disc is spinning at an average speed of OVER 10,000 miles per hour?) Second, the access time of flash memory is measured in nanoseconds, thus, there is no latency compared to hard disc, which typically has an access time measured in miniseconds. Some new generations of USB 2.0 compliant compact flash sticks can write faster than read (and can read and write simultaneously). While the write speed is currently maxed out below 10 MB/s, the technology is evolving very rapidly. If a market exists, a single channel USB 2.0 compliant CF disc may reach 60 MB/s. This is similar to the speed of ATA/66. Eventually, who knows, there may be multiple-channel CF discs. And that will be a totally different story. Wayne I am very suprised no one sells PC like this. All of the ones I found were cash registers or the like, no general purpose PC's. I would think this would be great for routers, firewalls, etc. High high availabilty stuff. Given all of the limitations that have been described, I don't see how it's all that useful to use CF in a high-availability environment like a router or firewall. PCs are cheap enough now that you can deploy a pair of fully-redundant PCs running Linux (or any other Unix-ish OS), on relatively fast hard drives, for a few hundred dollars. Why would you do anything else? Why bother with the added complexity of CF?
Re: [luau] No hard drive, only compact flash card
Mark Pettit wrote: I am very suprised no one sells PC like this. All of the ones I found were cash registers or the like, no general purpose PC's. I would think this would be great for routers, firewalls, etc. High high availabilty stuff. Given all of the limitations that have been described, I don't see how it's all that useful to use CF in a high-availability environment like a router or firewall. You fail to realize how little hard drive usage a router has :) My company, Computer Experts (www.comp-exp.com, call 317-833-3000 if you're interested), sells a router based on CF card (for either your PC or we're working with Single Board Computers these days as well) where the entire root filesystem is readonly. Yup, that's right (not write :). The entire root filesystem is never mounted readwrite. /tmp is on a tmpfs. /var can be mounted on a tmpfs if no hard drive is available for logging (and of course logs can be sent to another machine via standard syslog protocols). /dev is mounted in a small tmpfs as well and populated from a small tarball. The beauty of this is that you can actually treat it like a dumb router :) Most people wouldn't think of just unplugging a Linux box when they have to move it, but with our product you can! Because the entire fs is readonly, there's absoultely no issues with just unplugging it (other than losing state, which means all your connections will probably be reset if NAT is being used). Sorry if this sounds like an advertisement (it kinda is), but it's a really cool device! Did I mention all this is done off a 32MB flash card (so there's plenty of room for more functionality in the firmware...of course it already speaks SNMP). PCs are cheap enough now that you can deploy a pair of fully-redundant PCs running Linux (or any other Unix-ish OS), on relatively fast hard drives, for a few hundred dollars. Why would you do anything else? Why bother with the added complexity of CF? Added complexity of CF? You must have never set up a set of fully redundant systems :) I assure you, CF cards are far from complex! They're standard ATA flash devices. If you need to write to them, you can. But they really shine in situations like a router because most stuff isn't hard drive bound (only admin activities, logging, and config changes require accessing the hard drive at all!) and they're VERY reliable. Would you ever think of just dropping a hard drive on the ground? If you did it by accident would you be likely to ahve 100% data recovery? I've dropped tons of CF cards on the ground (and a computer one was in, d'oh!) before and they keep working fine. Reapeat after me: No moving parts == good thing :) Of course, CF cards aren't great for every day use. Their write speeds are VERY slow, and their read speeds aren't exactly fast (well, they would have been about 10 years ago :). Most situations will never hit the erase cycle limit, but there is always that concern as well in very dynamic environments (though Linux's fs cachcing can actually help quite a bit here with stuff that's REALLY dynamic like SQL databases, though I still wouldn't run an SQL DB on a CF card unless you could cache the whole thign in RAM). --MonMotha
Re: [luau] No hard drive, only compact flash card
Hawaii Linux Institute wrote: ... Second, the access time of flash memory is measured in nanoseconds, thus, there is no latency compared to hard disc, which typically has an access time measured in miniseconds. Milliseconds you mean. But I keep forgetting about this BIG one. In situations where sustained throughput is low, but random access are high, CF cards may actually be FASTER. A flash read cycle always takes the same amount of time (generally on the order of 30-120ns for NOR flash, not sure about NAND whcih many of the larger CF cards may use), no matter where you're accessing relative to where you just accessed. Hard drives are great for sequential access, but really suck on highly random small reads (say only a few bytes) because they have a moving head that has to seek. Seek times for consumer IDE hard drives seem to be running around 5-20ms these days. Some new generations of USB 2.0 compliant compact flash sticks can write faster than read (and can read and write simultaneously). While the write speed is currently maxed out below 10 MB/s, the technology is evolving very rapidly. If a market exists, a single channel USB 2.0 compliant CF disc may reach 60 MB/s. This is similar to the speed of ATA/66. Eventually, who knows, there may be multiple-channel CF discs. And that will be a totally different story. I'd like to know how this is technologically possible. Assuming NOR flash (I will admit up front that NAND flash may be MUCH faster) with a total cycle time (including address set up and data read strobe) of 30ns (an estimate, but the range is big, I think this is on the low end) with a 32bit wide data bus (mind you, the CF data bus is only 12 bits wide), and assuming that the bus and CF cycles are always in sync (they're not), we have a max theoretical of abotu 128MB/sec. This is assuming a pretty fast chip with no overhead and NO ERASE CYCLES (which take a LONG time). Writes generally involve erase cycles (unless they've gotten smart and erase things during idle time). It is posisble, but VERY unlikely that you would hit 60MB/sec. Are you sure this isn't the theoretical bandwidth of the bus? I'm not denying that it exists, just questioning the feasbility of it given current flash technology). I mostly work with just the ATA ones, not the USB ones as well (because they're cheap, and I'm using them as ATA devices :). Wayne --MonMotha
Re: [luau] No hard drive, only compact flash card
Here's a cool example of how this would work in a small half-width 1U server: http://www.ant-computing.com/ You can make your own, but it's not easy, instructions on how they did it are on the site. I am very suprised no one sells PC like this. All of the ones I found were cash registers or the like, no general purpose PC's. I would think this would be great for routers, firewalls, etc. High high availabilty stuff. Given all of the limitations that have been described, I don't see how it's all that useful to use CF in a high-availability environment like a router or firewall. But routers and firewalls run almost completely in RAM, if it had to touch the drive at every network packet, that would bog down any network considerably. I think the utility of the CF reader in high availability applications is totally legit, considering it would mainly be used as a means of storing an operating system to boot up in between shut downs. Now if CF was used to _replace_ RAM(as in swap), then the limitations would certainly bring the system to it's knees in high-availability situations. If it was a small little web server, I think it would be cool to have for a little LAN webpage or whatnot. I was told once the best minimalist distro was debian. I sure like the functionality of apt-get. A little off-topic, but if you like debian's apt-get, you'll also enjoy gentoo's http://gentoo.org emerge utility. This is off topic, because gentoo compiles all applications upon installation with emerge, which is quite uneconomical with a flash card system, due to lack of resources. But gentoo is my recent favorite distro, because you have complete control over what's going on, and you can optimize every application's compile for your system. It's not for the feint of heart though, because the installation is completely manual, but it does have step-by-step documentation, which is really useful in learning about how Linux works, at least it has been for me. -Kekoa
Re: [luau] No hard drive, only compact flash card
Thanks, you're right. It should be milliseconds, not miniseconds. :-) The technology on flash memory is evolving so rapidly, no one really knows what we could expect. For example, go to the USPTO web site and do a quick and dirty search on flash memory and Liauh, and you will find at least 11 patents. There are always those little bitty things that are constantly invented to improve the read, and more particularly, write, speed. Remember 7 or 8 years ago when we thought the EDO DRAM has reached the possible speed limit of dynamic memory cells? Current flash memory is almost at the same stage of the EDO DRAM. BTW, I did wrong calculation in the linear speed of a hard disc. It should be about 40 miles per hours, and definitely not 10,000 miles per hour. MonMotha wrote: Hawaii Linux Institute wrote: ... Second, the access time of flash memory is measured in nanoseconds, thus, there is no latency compared to hard disc, which typically has an access time measured in miniseconds. Milliseconds you mean. But I keep forgetting about this BIG one. In situations where sustained throughput is low, but random access are high, CF cards may actually be FASTER. A flash read cycle always takes the same amount of time (generally on the order of 30-120ns for NOR flash, not sure about NAND whcih many of the larger CF cards may use), no matter where you're accessing relative to where you just accessed. Hard drives are great for sequential access, but really suck on highly random small reads (say only a few bytes) because they have a moving head that has to seek. Seek times for consumer IDE hard drives seem to be running around 5-20ms these days. Some new generations of USB 2.0 compliant compact flash sticks can write faster than read (and can read and write simultaneously). While the write speed is currently maxed out below 10 MB/s, the technology is evolving very rapidly. If a market exists, a single channel USB 2.0 compliant CF disc may reach 60 MB/s. This is similar to the speed of ATA/66. Eventually, who knows, there may be multiple-channel CF discs. And that will be a totally different story. I'd like to know how this is technologically possible. Assuming NOR flash (I will admit up front that NAND flash may be MUCH faster) with a total cycle time (including address set up and data read strobe) of 30ns (an estimate, but the range is big, I think this is on the low end) with a 32bit wide data bus (mind you, the CF data bus is only 12 bits wide), and assuming that the bus and CF cycles are always in sync (they're not), we have a max theoretical of abotu 128MB/sec. This is assuming a pretty fast chip with no overhead and NO ERASE CYCLES (which take a LONG time). Writes generally involve erase cycles (unless they've gotten smart and erase things during idle time). It is posisble, but VERY unlikely that you would hit 60MB/sec. Are you sure this isn't the theoretical bandwidth of the bus? I'm not denying that it exists, just questioning the feasbility of it given current flash technology). I mostly work with just the ATA ones, not the USB ones as well (because they're cheap, and I'm using them as ATA devices :). Wayne --MonMotha ___ LUAU mailing list [EMAIL PROTECTED] http://videl.ics.hawaii.edu/mailman/listinfo/luau .
Re: [luau] No hard drive, only compact flash card
Hawaii Linux Institute wrote: BTW, I did a wrong calculation in the linear speed of a hard disc. It should be about 40 miles per hours, and definitely not 10,000 miles per hour. But keep in mind that the separation between the magnetic head (which is stationary) and the disc surface (which is spinning at about 40 MPH) is less than 20 Angstroms. Basically, this separation is filled with two or at most three layers of lubricating molecules. These lubricating molecules, which are mobile, can disappear due to heat dissipation, pressure gradients, or whatever reason(s). Wayne
