Re: HDD vs. RAID (was Re: Lilo Q)
On Mon, 2002-06-10 at 14:22, Robert Webb wrote: > > > Dave Sherohman wrote: [snip] > yup, there IDE RAID controllers out there. Whether they help speed or > not I don't know. > I am actually running one that does only RAID 1 for redundancy for my > firewall. Cannot > afford to have that go down. :-) I think, if I wanted that type of security in my firewall, that I'd create a CD-R + floppy disk solution, where the floppy disk has just one file: the firewall config script. -- +-+ | Ron Johnson, Jr.Home: [EMAIL PROTECTED] | | Jefferson, LA USA http://ronandheather.dhs.org:81 | | | | "I have created a government of whirled peas..."| | Maharishi Mahesh Yogi, 12-May-2002, | ! CNN, Larry King Live | +-+ -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of "unsubscribe". Trouble? Contact [EMAIL PROTECTED]
Re: HDD vs. RAID (was Re: Lilo Q) - fw
hi ya robert why not have two firewalls ??? ( 2x pentium-90Mhz for example .. something cheap, but fast enough) +-- fw1 --+ internet -> csu/dus -> hub -> + +--> hub -> your lan +-- fw2 --+ when one goes down use the other ... ( we are ignoring hubbs as being the single pointof failures ( and the csu/dsu ... On Mon, 10 Jun 2002, Robert Webb wrote: > yup, there IDE RAID controllers out there. Whether they help speed or > not I don't know. > I am actually running one that does only RAID 1 for redundancy for my > firewall. Cannot > afford to have that go down. :-) > -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of "unsubscribe". Trouble? Contact [EMAIL PROTECTED]
Re: HDD vs. RAID (was Re: Lilo Q)
On Mon, 2002-06-10 at 13:39, Dave Sherohman wrote: > On Mon, Jun 10, 2002 at 12:07:22PM -0500, Ron Johnson wrote: > > The problem with JBODs (just big ole disks, i.e. single disks) > > JBOD = Just a Bunch Of Disks, i.e., several drives operating > independently. A JBOD can be organized into a RAID, but doesn't have > to be. We must work in different shops... > > With RAID solutions, the read-write heads > > will be in as many different places at once as you have disks. > > This is primarily a benefit in RAID0 or 5 configurations. RAID1 could > benefit from it also, but a lot of RAID implementations are too stupid > to take advantage of it. RAID4 loses some of this benefit due to the > limitations of having all the parity data on a single disk. If I remember correctly from the last time I created a RAID set, the docs said: WRITE: READ Fastest RAID1+0 RAID1+0 RAID0 RAID0 single RAID5 RAID1 RAID4 RAID4 RAID1 Slowest RAID5 single > > Note, though, that since the CPU overhead from calculating RAID[45] > > recovery blocks necessitates a caching controller. Otherwise, > > write speeds will be slower. > > RAID1 is also typically slower since the write isn't considered to be > complete until it has taken place on all disks (having read-write heads > in many places helps reads and hurts writes). In the last few years, when speed is imperative, we've bitten the cost bullet and gone with RAID1+0, mirrored stripesets (or striped mirrorsets; I don't know how the controller internally handles it). -- +-+ | Ron Johnson, Jr.Home: [EMAIL PROTECTED] | | Jefferson, LA USA http://ronandheather.dhs.org:81 | | | | "I have created a government of whirled peas..."| | Maharishi Mahesh Yogi, 12-May-2002, | ! CNN, Larry King Live | +-+ -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of "unsubscribe". Trouble? Contact [EMAIL PROTECTED]
Re: HDD vs. RAID (was Re: Lilo Q)
Dave Sherohman wrote: [snip] Any sort of true hardware RAID setup (beware the hybrids, since this doesn't apply to them) will interact with the rest of the system as a single device. The question of whether to put the individual drives on separate controllers or not doesn't apply, since the drives connect to the RAID controller and the RAID controller attaches to the SCSI (or IDE, I suppose, though I've never seen an IDE hardware RAID controller) bus as a single device. Even the BIOS thinks it's just one disk. yup, there IDE RAID controllers out there. Whether they help speed or not I don't know. I am actually running one that does only RAID 1 for redundancy for my firewall. Cannot afford to have that go down. :-) Robert -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of "unsubscribe". Trouble? Contact [EMAIL PROTECTED]
Re: HDD vs. RAID (was Re: Lilo Q)
On Mon, Jun 10, 2002 at 10:00:16AM -0400, Ian D. Stewart wrote: > So, the way I'm reading this, a RAID 5 stack w/ 5 20 GB hard drives > provides improved access speed and reliability at the cost of slightly > reduced storage. Yep. Different RAID levels are basically different tradeoffs between reliability and capacity (speed should theoretically be improved at all levels, so I don't see it as a major distinguishing factor, even though it's not an even increase at all levels in the real world). RAID0 is all about capacity, RAID1 is the ultimate in failure-tolerance, RAID5 and RAID0+1 are different ways of trying to strike a balance between the two. > An earlier thread was making reference to setting up > seperate controllers for each HDD. I have seen adverts for stand-alone > RAID towers. Would the use of one of these towers do away with the > need for seperate controllers, and if so do these towers support IDE or > just SCSI? Any sort of true hardware RAID setup (beware the hybrids, since this doesn't apply to them) will interact with the rest of the system as a single device. The question of whether to put the individual drives on separate controllers or not doesn't apply, since the drives connect to the RAID controller and the RAID controller attaches to the SCSI (or IDE, I suppose, though I've never seen an IDE hardware RAID controller) bus as a single device. Even the BIOS thinks it's just one disk. Keep in mind, though, that my earlier comment about needing a seperate controller for each drive is IDE-specific. SCSI controllers are a lot smarter about handling multiple devices and can deal with this more effectively, just so long as you verify that the SCSI controller has enough bandwidth for all the drives to transfer data at their maximum rates. -- When we reduce our own liberties to stop terrorism, the terrorists have already won. - reverius Innocence is no protection when governments go bad. - Tom Swiss -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of "unsubscribe". Trouble? Contact [EMAIL PROTECTED]
Re: HDD vs. RAID (was Re: Lilo Q)
On Mon, Jun 10, 2002 at 12:07:22PM -0500, Ron Johnson wrote: > The problem with JBODs (just big ole disks, i.e. single disks) JBOD = Just a Bunch Of Disks, i.e., several drives operating independently. A JBOD can be organized into a RAID, but doesn't have to be. > With RAID solutions, the read-write heads > will be in as many different places at once as you have disks. This is primarily a benefit in RAID0 or 5 configurations. RAID1 could benefit from it also, but a lot of RAID implementations are too stupid to take advantage of it. RAID4 loses some of this benefit due to the limitations of having all the parity data on a single disk. > Note, though, that since the CPU overhead from calculating RAID[45] > recovery blocks necessitates a caching controller. Otherwise, > write speeds will be slower. RAID1 is also typically slower since the write isn't considered to be complete until it has taken place on all disks (having read-write heads in many places helps reads and hurts writes). -- When we reduce our own liberties to stop terrorism, the terrorists have already won. - reverius Innocence is no protection when governments go bad. - Tom Swiss -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of "unsubscribe". Trouble? Contact [EMAIL PROTECTED]
Re: HDD vs. RAID (was Re: Lilo Q)
On Mon, Jun 10, 2002 at 09:46:45AM -0400, Ian D. Stewart wrote: > So then, the primary advantages of RAID are access speed and data > redundancy The primary advantages of RAID are highly dependent on what flavor of RAID you're using. RAID0 and RAID1, e.g., are practically the opposite of each other in terms of speed, capacity, and reliability. Reference my earlier post in this thread for further details. > and the primary advantage of a stand-alone HDD is > reliability? A single drive is less likely to have a hardware failure (since there's less hardware to fail), but, if it does fail, you lose access to the data. RAIDs tend to have more frequent hardware failures, but, unless it's a RAID0, your data will remain available despite the failure. -- When we reduce our own liberties to stop terrorism, the terrorists have already won. - reverius Innocence is no protection when governments go bad. - Tom Swiss -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of "unsubscribe". Trouble? Contact [EMAIL PROTECTED]
Re: HDD vs. RAID (was Re: Lilo Q)
On Mon, 2002-06-10 at 08:46, Ian D. Stewart wrote: > On 2002.06.10 03:35 Anthony DeRobertis wrote: > > On Sun, 2002-06-09 at 20:33, Alvin Oga wrote: > > > > > if you have a nearly full 80GB disks ... it wont matter > > > if you have 1x 80GB or 4x 20GB( stripping ) > > > > No, it does matter. You can expect at least one of four 20GB drives to > > fail much sooner than one 80GB drive, assuming same MTBF numbers on > > all > > drives. > > > > The MTBF for one 50,000hr MTBF disk is 50,000hr. For four of them, it > > is > > 13,500Hr. > > > > [ And, if you operate the four for a year, you can expect 1 to fail. ] > > So then, the primary advantages of RAID are access speed and data > redundancy and the primary advantage of a stand-alone HDD is > reliability? Well, redundancy is one way of ensuring reliability, so your statement is slightly off. The problem with JBODs (just big ole disks, i.e. single disks) is that all of your eggs are in one basket. Even though the MTBF is 50,000 hours, the M in MTBF is, of course Mean, so even though it should last 50,000 hours, it _might_ puke tomorrow. So, use RAID[145] (or better yet: RAID1+0) just so you are spreading the risk... Another point that may have already been mentioned: with a JBOD, you only have one spindle, so the read-write heads can only be at 1 place at a time. With RAID solutions, the read-write heads will be in as many different places at once as you have disks. Note, though, that since the CPU overhead from calculating RAID[45] recovery blocks necessitates a caching controller. Otherwise, write speeds will be slower. -- +-+ | Ron Johnson, Jr.Home: [EMAIL PROTECTED] | | Jefferson, LA USA http://ronandheather.dhs.org:81 | | | | "I have created a government of whirled peas..."| | Maharishi Mahesh Yogi, 12-May-2002, | ! CNN, Larry King Live | +-+ -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of "unsubscribe". Trouble? Contact [EMAIL PROTECTED]
Re: HDD vs. RAID (was Re: Lilo Q)
On 2002.06.10 05:48 Anthony DeRobertis wrote: On Mon, 2002-06-10 at 03:46, Alvin Oga wrote: > > - and if the drives gonna fail... i say its more likely to die > within the first 30 days ... Yes. MTBF only measures how likely it is to fail during the middle of its life. A good number die early (defective) and late (worn out). Not many die in the middle. That's what MTBF measures. I was speaking of the MTBF of RAID-0 where any one disk death means the whole array is gone. >- what's the likelyhood of 2 drives that fail ... >rendering the raid subsystem to be just blank disks.. Not much. Especially if you replace the failed disk promptly, or have a spare. >( hopefully one can rest a little better after the first disk >( dies... or is more of the same fate to happen to the rest of >( the disks ... Neither. Unless the failure was due to the environment (e.g., running disks at 120 degress in a paint can shaker), having one fail makes others neither more likely nor less likely to fail. > > - i still prefer 1 large disks.. instead of many small ones... If you have many small disks and one fails, you are OK, as long as you used RAID 1 or RAID 4/5. You can replace the one failed disk. If your one large disk fails, you're down until you restore from backups. So, the way I'm reading this, a RAID 5 stack w/ 5 20 GB hard drives provides improved access speed and reliability at the cost of slightly reduced storage. An earlier thread was making reference to setting up seperate controllers for each HDD. I have seen adverts for stand-alone RAID towers. Would the use of one of these towers do away with the need for seperate controllers, and if so do these towers support IDE or just SCSI? Thanx for all the input. I'm finding all of this info very interesting! Ian -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of "unsubscribe". Trouble? Contact [EMAIL PROTECTED]
Re: HDD vs. RAID (was Re: Lilo Q)
On 2002.06.10 03:35 Anthony DeRobertis wrote: On Sun, 2002-06-09 at 20:33, Alvin Oga wrote: > if you have a nearly full 80GB disks ... it wont matter > if you have 1x 80GB or 4x 20GB( stripping ) No, it does matter. You can expect at least one of four 20GB drives to fail much sooner than one 80GB drive, assuming same MTBF numbers on all drives. The MTBF for one 50,000hr MTBF disk is 50,000hr. For four of them, it is 13,500Hr. [ And, if you operate the four for a year, you can expect 1 to fail. ] So then, the primary advantages of RAID are access speed and data redundancy and the primary advantage of a stand-alone HDD is reliability? Thanx, Ian -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of "unsubscribe". Trouble? Contact [EMAIL PROTECTED]
Re: HDD vs. RAID (was Re: Lilo Q)
On Mon, 2002-06-10 at 03:46, Alvin Oga wrote: > > - and if the drives gonna fail... i say its more likely to die > within the first 30 days ... Yes. MTBF only measures how likely it is to fail during the middle of its life. A good number die early (defective) and late (worn out). Not many die in the middle. That's what MTBF measures. I was speaking of the MTBF of RAID-0 where any one disk death means the whole array is gone. > - what's the likelyhood of 2 drives that fail ... > rendering the raid subsystem to be just blank disks.. Not much. Especially if you replace the failed disk promptly, or have a spare. > ( hopefully one can rest a little better after the first disk > ( dies... or is more of the same fate to happen to the rest of > ( the disks ... Neither. Unless the failure was due to the environment (e.g., running disks at 120 degress in a paint can shaker), having one fail makes others neither more likely nor less likely to fail. > > - i still prefer 1 large disks.. instead of many small ones... If you have many small disks and one fails, you are OK, as long as you used RAID 1 or RAID 4/5. You can replace the one failed disk. If your one large disk fails, you're down until you restore from backups. > > - if the server needs to stay up 24x7 ... than i'd like to have 2 or 3 > servers to be looking like 1 server... Yep. This isn't always easy, though. signature.asc Description: This is a digitally signed message part
Re: HDD vs. RAID (was Re: Lilo Q)
hi ya anthony yes... good point on MTBF... - and if the drives gonna fail... i say its more likely to die within the first 30 days ... ( some disks more likely to die than others irrespective of the MTBF and name-brands.. - i have a pile of "bad/flaky IBM disks" ... about 1-5% failure rates (basically not good as one would expect) - but if one does have 4 drives raid5 and a disk dies.. thats still recoverable and you're still limping along until you can replace that dead disk and get back to "normal operation"... - what's the likelyhood of 2 drives that fail ... rendering the raid subsystem to be just blank disks.. ( hopefully one can rest a little better after the first disk ( dies... or is more of the same fate to happen to the rest of ( the disks ... - i still prefer 1 large disks.. instead of many small ones... - if the server needs to stay up 24x7 ... than i'd like to have 2 or 3 servers to be looking like 1 server... magic... c ya alvin On 10 Jun 2002, Anthony DeRobertis wrote: > On Sun, 2002-06-09 at 20:33, Alvin Oga wrote: > > > if you have a nearly full 80GB disks ... it wont matter > > if you have 1x 80GB or 4x 20GB( stripping ) > > No, it does matter. You can expect at least one of four 20GB drives to > fail much sooner than one 80GB drive, assuming same MTBF numbers on all > drives. > > The MTBF for one 50,000hr MTBF disk is 50,000hr. For four of them, it is > 13,500Hr. > > [ And, if you operate the four for a year, you can expect 1 to fail. ] > > > best best... > > === > > === backup data regularly to DIFFERENT systems .. > > === > > Or tape. But whatever you do, make sure you: > >1) Test your ability to restore data. Do this regularly. You'd hate > it if you couldn't. >2) Verify your backups. Very important for tape. > > -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of "unsubscribe". Trouble? Contact [EMAIL PROTECTED]
Re: HDD vs. RAID (was Re: Lilo Q)
On Sun, 2002-06-09 at 20:33, Alvin Oga wrote: > if you have a nearly full 80GB disks ... it wont matter > if you have 1x 80GB or 4x 20GB( stripping ) No, it does matter. You can expect at least one of four 20GB drives to fail much sooner than one 80GB drive, assuming same MTBF numbers on all drives. The MTBF for one 50,000hr MTBF disk is 50,000hr. For four of them, it is 13,500Hr. [ And, if you operate the four for a year, you can expect 1 to fail. ] > best best... > === > === backup data regularly to DIFFERENT systems .. > === Or tape. But whatever you do, make sure you: 1) Test your ability to restore data. Do this regularly. You'd hate it if you couldn't. 2) Verify your backups. Very important for tape. signature.asc Description: This is a digitally signed message part
Re: HDD vs. RAID (was Re: Lilo Q)
hi ya fun stuff. it depends ... if you have a nearly full 80GB disks ... it wont matter if you have 1x 80GB or 4x 20GB( stripping ) - i rather worry about 1 large disk failure... than to worry about which of the 4 small disks gonna die ... also makes 4x the mess in power and cables...etc.. if you use raid1 ( mirroring ) effective usage is 1/2 of the total raw disk space - not useful if you're using 40gb of data on your 80gb disks - good option if you want to protect up to 40GB of data from disk crashes and want to stay online even if a single disk crashes as was suggested earlier... use raid5 instead.. if 4 disks raid5 ... if any one disk dies.. you can still recover ( effective disk space is 60GB out of 80gb ) if you add a 5th 20GB disks... you still have 80gb out of 100GB of total usable space still only protected against one disk failure but any of the 4 or 5 disks could die ... instead of one 80gb disk == if read transfer speed is important.. use raid0 ( stripping ) over raid1 ( mirror ) - you should be able to read data 2x as fast... but writing is a little slower... best best... === === backup data regularly to DIFFERENT systems .. === c ya alvin -- original bios/disk question ... - most BIOS can and does support up to 130GB or so w/o any problems 160GB ata 133 being the tricky disks to play with - most mb cannot boot from hde/hdf/hdg/hdh - so tell lilo to write mbr info to hda and that / is still /dev/hde ... works fine ... On Sun, 9 Jun 2002, Dave Sherohman wrote: > On Sun, Jun 09, 2002 at 09:35:05AM -0400, Ian D. Stewart wrote: ... > > (say, 80 GB hard drive vs. raid tower w/ 4 20 GB hard > > drives) ? > > If you're getting 80G from 4*20G drive, that must be a RAID 0, so > the RAID would give you a nice boost to data transfer rates, but > you'd better keep a current backup because if any one of those 4 > drives goes bad, you'll lose all your data. (OTOH, add a fifth 20G > drive, make it a RAID 5, and you'll have a winner.) > > Side note: Comments on performance assume that each drive is on a > separate IDE channel all by itself. If your 4 20G drives are hda, > hdb, hdc, and hdd, you're going to take a major performance hit. > Unlike SCSI, IDE can't run two drives efficiently on the same > channel. > > -- > When we reduce our own liberties to stop terrorism, the terrorists > have already won. - reverius > > Innocence is no protection when governments go bad. - Tom Swiss > > > -- > To UNSUBSCRIBE, email to [EMAIL PROTECTED] > with a subject of "unsubscribe". Trouble? Contact [EMAIL PROTECTED] > -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of "unsubscribe". Trouble? Contact [EMAIL PROTECTED]
Re: HDD vs. RAID (was Re: Lilo Q)
"Ian D. Stewart" wrote: > As the size of IDE hard drives increase, what are the > advantages/disadvantages of using a single large hard drive as opposed > to a RAID stack (say, 80 GB hard drive vs. raid tower w/ 4 20 GB hard > drives) ? I'd say it all depends on the specs of hard drives. If you're comparing apples to apples (i.e. the same or very similar models) then the RAID solution should always be faster. When you start comparing older to newer, I'm sure this clouds the situation. Also realize that with IDE, only one device on an interface can talk at once. So if you are doing RAID with IDE drives, you really have to have each drive on its own interface in order to see a speedup. So this does affect the price equation a bit. If you're running out of controllers, you could put a CDROM as the slave to a HD master. As long as the CDROM supports DMA mode, it will not slow down the HD at all. You can mix DMA modes on an interface, but you cannot mix PIO and DMA. So if you put an older CDROM that only supports PIO as a slave to an UltraDMA HD, the HD will not be able to use DMA mode. And of course, expect a slowdown if you try to use the CDROM and HD at the same time. Brian -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of "unsubscribe". Trouble? Contact [EMAIL PROTECTED]
Re: HDD vs. RAID (was Re: Lilo Q)
On Sun, Jun 09, 2002 at 09:35:05AM -0400, Ian D. Stewart wrote: > As the size of IDE hard drives increase, what are the > advantages/disadvantages of using a single large hard drive as opposed > to a RAID stack Well, that depends on what flavor of RAID you're talking about... In general: RAID 0: Capacity = sum of all disks, improved performance, reduced reliability (if one drive fails, everything goes with it) RAID 1: Capacity = smallest disk, read performance could improve but is usually not affected, write performance tends to be slower, best reliability (as long as one disk survives, your data is OK) RAID 5: Capacity = smallest disk * (number of disks - 1), improved perfomance, data can survive failure of one disk, but all is lost if a second disk fails before the contents of the first are regenerated RAID 0+1: Capacity = smallest disk * (number of disks / 2), good performance, good reliability > (say, 80 GB hard drive vs. raid tower w/ 4 20 GB hard > drives) ? If you're getting 80G from 4*20G drive, that must be a RAID 0, so the RAID would give you a nice boost to data transfer rates, but you'd better keep a current backup because if any one of those 4 drives goes bad, you'll lose all your data. (OTOH, add a fifth 20G drive, make it a RAID 5, and you'll have a winner.) Side note: Comments on performance assume that each drive is on a separate IDE channel all by itself. If your 4 20G drives are hda, hdb, hdc, and hdd, you're going to take a major performance hit. Unlike SCSI, IDE can't run two drives efficiently on the same channel. -- When we reduce our own liberties to stop terrorism, the terrorists have already won. - reverius Innocence is no protection when governments go bad. - Tom Swiss -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of "unsubscribe". Trouble? Contact [EMAIL PROTECTED]
