CCD on /
I've been searching google if the OpenBSD kernel can boot off CCD. Is this supported? My requirements for setting up any new servers are that they have fault tolerant disks. In this case, there is no option for hardware RAID because it's a blade. Can OpenBSD boot off RAIDFrame? Thanks
Re: CCD on /
Rob Foster on 2005-06-11 16:49:09 -0400: > I've been searching google if the OpenBSD kernel can boot off CCD. Is > this supported? > > My requirements for setting up any new servers are that they have > fault tolerant disks. In this case, there is no option for hardware > RAID because it's a blade. Can OpenBSD boot off RAIDFrame? I can't speak for CCD, but I have successfully booted OpenBSD off RAIDFrame in a RAID1 configuration.
Re: CCD on /
but you are not supposed to. On Jun 11, 2005, at 4:47 PM, Alec Berryman wrote: Rob Foster on 2005-06-11 16:49:09 -0400: I've been searching google if the OpenBSD kernel can boot off CCD. Is this supported? My requirements for setting up any new servers are that they have fault tolerant disks. In this case, there is no option for hardware RAID because it's a blade. Can OpenBSD boot off RAIDFrame? I can't speak for CCD, but I have successfully booted OpenBSD off RAIDFrame in a RAID1 configuration.
Re: CCD on /
Marco Peereboom on 2005-06-11 17:20:28 -0500: > but you are not supposed to. raidctl(8) gives an example of how to set up root or RAID with RAIDFrame and does not discourage the reader from doing so. "At this time, RAID 0, 1, 4, and 5 sets are all supported as root devices." > On Jun 11, 2005, at 4:47 PM, Alec Berryman wrote: > > >Rob Foster on 2005-06-11 16:49:09 -0400: > > > > > >>I've been searching google if the OpenBSD kernel can boot off CCD. Is > >>this supported? > >> > >>My requirements for setting up any new servers are that they have > >>fault tolerant disks. In this case, there is no option for hardware > >>RAID because it's a blade. Can OpenBSD boot off RAIDFrame? > >> > > > >I can't speak for CCD, but I have successfully booted OpenBSD off > >RAIDFrame in a RAID1 configuration.
Re: CCD on /
Marco Peereboom on 2005-06-11 17:20:28 -0500: but you are not supposed to. raidctl(8) gives an example of how to set up root or RAID with RAIDFrame and does not discourage the reader from doing so. "At this time, RAID 0, 1, 4, and 5 sets are all supported as root devices." I've succesfully set up raidframe (raid 1 i think) on root once, but would advice against it, since it makes upgrading a little difficult, since raidframe is not in generic, but it can be done (i did it back in the 3.1 days). /jtm On Jun 11, 2005, at 4:47 PM, Alec Berryman wrote: >Rob Foster on 2005-06-11 16:49:09 -0400: > > >>I've been searching google if the OpenBSD kernel can boot off >>CCD. Is >>this supported? >> >>My requirements for setting up any new servers are that they have >>fault tolerant disks. In this case, there is no option for >>hardware >>RAID because it's a blade. Can OpenBSD boot off RAIDFrame? >> > >I can't speak for CCD, but I have successfully booted OpenBSD off >RAIDFrame in a RAID1 configuration.
Re: CCD on /
Why is RAIDFrame not in the generic kernel? Is it too big, or buggy? On 6/11/05, Jens Teglhus Mxller <[EMAIL PROTECTED]> wrote: > > Marco Peereboom on 2005-06-11 17:20:28 -0500: > > > >> but you are not supposed to. > > > > raidctl(8) gives an example of how to set up root or RAID with > > RAIDFrame and does not discourage the reader from doing so. > > > > "At this time, RAID 0, 1, 4, and 5 sets are all supported as root > > devices." > > I've succesfully set up raidframe (raid 1 i think) on root once, but > would advice against it, since it makes upgrading a little difficult, > since raidframe is not in generic, but it can be done (i did it back > in the 3.1 days). > > /jtm > > >> On Jun 11, 2005, at 4:47 PM, Alec Berryman wrote: > >> > >> >Rob Foster on 2005-06-11 16:49:09 -0400: > >> > > >> > > >> >>I've been searching google if the OpenBSD kernel can boot off > >> >>CCD. Is > >> >>this supported? > >> >> > >> >>My requirements for setting up any new servers are that they have > >> >>fault tolerant disks. In this case, there is no option for > >> >>hardware > >> >>RAID because it's a blade. Can OpenBSD boot off RAIDFrame? > >> >> > >> > > >> >I can't speak for CCD, but I have successfully booted OpenBSD off > >> >RAIDFrame in a RAID1 configuration.
Re: CCD on /
Rob Foster wrote: Why is RAIDFrame not in the generic kernel? Is it too big, or buggy? It makes the kernel 500K bigger. I think that's in the FAQ somewhere. :) -- No virus found in this outgoing message. Checked by AVG Anti-Virus. Version: 7.0.323 / Virus Database: 267.6.9 - Release Date: 11/06/2005
Re: CCD on /
just thoughts... > It makes the kernel 500K bigger. I think that's in the FAQ somewhere. :) but really, would 500K hurt? who has an x86, that does not survive the waste of 500K RAM (even my 486DX had 64MB)? are there any plans to support the kernel being loaded directly from a RAID partition in order to avoid that annoying mini-boot-partition which cannot be raided? br, mdff...
Re: CCD on /
Making, drinking tea and reading an opus magnum from mdff: > just thoughts... > > > It makes the kernel 500K bigger. I think that's in the FAQ somewhere. :) > but really, would 500K hurt? who has an x86, that does not > survive the waste of 500K RAM (even my 486DX had 64MB)? it does not pay off > are there any plans to support the kernel being loaded > directly from a RAID partition in order to avoid that > annoying mini-boot-partition which cannot be raided? and what are you going to do in case the raid partition itself gets broken? how are you going to repair if you cannot boot the machine w/o any additional hardware attached? cu -- paranoic mickey (my employers have changed but, the name has remained)
Re: CCD on /
from "http://www.cs.usask.ca/staff/oster/raid.html"; " Q Why are kernels with RAIDframe so huge? A RAIDframe is not small. It typically requires at least 500K of extra space [May 24, 2000?: Gee.. this is old too... on i386, I believe the additional space required is only about 320K these days. Feb. 22, 2004: RAIDframe weighs in at 150K for i386]. This makes it tight to get it working on something like a Sun 3/50 (kernels must be less than 1MB in size), but if you're going to be running this on a server, chances are an extra 500K isn't going to be much of a problem. Doing RAID well and with as many options as RAIDframe provides takes a lot of space. I am working on pruning out unnecesary bits from the driver. " is openbsd using this code? On 6/12/05, James Herbert <[EMAIL PROTECTED]> wrote: > Rob Foster wrote: > > Why is RAIDFrame not in the generic kernel? Is it too big, or buggy? > > It makes the kernel 500K bigger. I think that's in the FAQ somewhere. :) > > > -- > No virus found in this outgoing message. > Checked by AVG Anti-Virus. > Version: 7.0.323 / Virus Database: 267.6.9 - Release Date: 11/06/2005
Re: CCD on /
mdff wrote: > are there any plans to support the kernel being loaded > directly from a RAID partition in order to avoid that > annoying mini-boot-partition which cannot be raided? WHY do you want to mirror root? keep in mind, OpenBSD has an "altroot" system in place, which will update your backup root partition every night with any changes that took place over the day on your root partition. What are you storing on root that is changing more often than once a day? Software mirroring is cheap..but it causes problems, too. It takes time to rebuild, it complicates upgrades, performance either sucks or is non-existant when rebuilding. In general, you probably only want to mirror the data you have that is actively changing continually. I spent a bit of time trying out ccd(4) mirroring recently. I worked hard to build a system where as much of a disk was mirrored as I could -- I had a root partition of 130M, and had the rest of the drive be a big mirrored ccd(4) partition which I then sub-divided. It worked... But after I set up the altroot backup system, I realized...do I really CARE if my /usr partition is mirrored up? How about my /etc files? Isn't a nightly sync more than sufficient? Not only that, if I hose something, I have a whole drive that HASN'T been messed up yet (hate to use the word "backup" with a non-rotated, non-removable media. Let's call it "Second chance"). If I'm running a mail server, I want a mirror of my mail spools, of course. If I'm running a webserver, I might want a mirror of my data and my logs. Firewall? Maybe my logs, but really, why do I need a mirror there at all? Oh, sure, you might want your system to stay running after it wuffs a drive, but if you are running an IDE system, it almost certainly won't. If you are running SCSI, it *might*, but don't count on it. Consider cheap (i.e., software) RAID systems a way to rapidly repair a broken computer, not a way to keep the system running without interruption. BTW: one other reason RAIDframe isn't in GENERIC is you have to customize your kernel in other ways, not just turning it on. From raid(4): "It is important that drives be hard-coded at their respective addresses (i.e., not left free-floating, where a drive with SCSI ID of 4 can end up as /dev/sd0c) for well-behaved functioning of the RAID device. This is true for all types of drives, including IDE, HP-IB, etc." (ccd(4), due to its simplicity, doesn't seem to care). Nick.
Re: CCD on /
On Wed, Jun 15, 2005 at 09:19:26PM -0400, Nick Holland wrote: > BTW: one other reason RAIDframe isn't in GENERIC is you have to > customize your kernel in other ways, not just turning it on. From raid(4): > "It is important that drives be hard-coded at their respective addresses > (i.e., not left free-floating, where a drive with SCSI ID of 4 can end > up as /dev/sd0c) for well-behaved functioning of the RAID device. This > is true for all types of drives, including IDE, HP-IB, etc." With the new raid autoconfiguration setup this is not the case.
Re: CCD on /
> and what are you going to do in case the raid partition > itself gets broken? how are you going to repair if you > cannot boot the machine w/o any additional hardware attached? therefore you'd have to setup an explicit non-raided partition or hdd with a repair-root on all servers with raid-controllers being supported by openbsd. such a "repair-partition" would be a a nice feature, but it shouldn't be inevitable. if one needs such a functionality, he'd even be able to realize it with nw- boot. my understanding of raid1 is avoiding loss of data because of hw-errors. i don't see raid1 as a fault-tolerance for kernels, users, raid- or fs-code not working correctly. finally, the current way to realize raid1 under openbsd makes each install and upgrade much more difficult than it could be. in the worst case you have to take care for 3 basic os setups, their up-to-dateness and security on one machine. br, mdff...
Re: CCD on /
> Oh, sure, you might want your system to stay running after it wuffs a > drive, but if you are running an IDE system, it almost certainly won't. > If you are running SCSI, it *might*, but don't count on it. Consider > cheap (i.e., software) RAID systems a way to rapidly repair a broken > computer, not a way to keep the system running without interruption. Why wouldn't a two drive ATA/SATA system which was raidframe mirrored stay up if one of the drives went belly up? I've been spending some cycles automating the kernel build/raidframe configure process assuming it was worth the extra effort
Re: CCD on /
On Thu, Jun 16, 2005 at 03:11:46PM -0400, Nick Bender wrote: > Why wouldn't a two drive ATA/SATA system which was raidframe mirrored > stay up if one of the drives went belly up? I've been spending some > cycles automating the kernel build/raidframe configure process > assuming it was worth the extra effort Controllers don't tend to like it. Sometimes with disk failure, the controller will fail too!
Re: CCD on /
responding 2 nick: > WHY do you want to mirror root? i do not like software raid at all and i even more dislike ide-raid (regardless whether it's sw or any other hw-raid). but sometimes you use hw where an hw-raid is not supported (even if there's a controller on-board...) and that's my reason for trying and using sw-raid like raidframe because it is better than nothing... really talking 'bout raid1 like i want it to be: having redundant power, redundant disks w hotplug, scsi320 at least and a good raid-controller-hw with a really smart bios handling any problems... i totally agree with you that scsi-hw-raid w hotplug support is definitely what you want on any production server. but if it comes to a development server or anything you just test on, you'd want to mirror root also, because you don't have to take care for your "2nd chance". you just throw the damn bad ide-hdd out of the window and the other one would bring up your system again like it was 10 minutes (add time for hangup, shutdown and some coffee) ago.
Re: CCD on /
At 08:31 PM 6/16/2005 +0100, Niall O'Higgins wrote: Controllers don't tend to like it. Sometimes with disk failure, the controller will fail too! The ASUS A7V880 runs just fine with one disk dead - infant mortality a few months ago. Lee
Re: CCD on /
exactly. I'm using blades with no hardware raid controller. software raid that worked just like hardware raid would be the best solution until we get better hardware. On 6/16/05, mdff <[EMAIL PROTECTED]> wrote: > responding 2 nick: > > > WHY do you want to mirror root? > > i do not like software raid at all and i even more dislike > ide-raid (regardless whether it's sw or any other hw-raid). > but sometimes you use hw where an hw-raid is not supported > (even if there's a controller on-board...) and that's my > reason for trying and using sw-raid like raidframe because > it is better than nothing... > really talking 'bout raid1 like i want it to be: having redundant > power, redundant disks w hotplug, scsi320 at least and a good > raid-controller-hw with a really smart bios handling any problems... > > i totally agree with you that scsi-hw-raid w hotplug support > is definitely what you want on any production server. > but if it comes to a development server or anything you just > test on, you'd want to mirror root also, because you don't > have to take care for your "2nd chance". you just throw the > damn bad ide-hdd out of the window and the other one would > bring up your system again like it was 10 minutes (add time for > hangup, shutdown and some coffee) ago.
Re: CCD on /
On Thursday 16 June 2005 20:31, Niall O'Higgins wrote: > On Thu, Jun 16, 2005 at 03:11:46PM -0400, Nick Bender wrote: > > Why wouldn't a two drive ATA/SATA system which was raidframe mirrored > > stay up if one of the drives went belly up? I've been spending some > > cycles automating the kernel build/raidframe configure process > > assuming it was worth the extra effort > > Controllers don't tend to like it. Sometimes with disk failure, the > controller will fail too! On the sort of hardware I'd consider running SATA RAID on, you've got at least two controllers, driving four or more ports. Blow a controller and a drive, and you're just waiting till you can afford a replacement drive and controller/motherboard. -- Simon Farnsworth [demime 1.01d removed an attachment of type application/pgp-signature]
Re: CCD on /
On 6/16/05, Niall O'Higgins <[EMAIL PROTECTED]> wrote: > Controllers don't tend to like it. Sometimes with disk failure, the > controller will fail too! Apart from that, you'll suffer from various annoying delays if for any reason parts in the system try to access the failed drive. Admittedly, I only saw this on 3.5 as it never occurred again. Nowadays, I'm a happy ami(4) user. For those who believe the observation above not to be a surprise: they're probably right. In my defence, that's I didn't report it. Cheers, Rogier -- If you don't know where you're going, any road will get you there.
Re: CCD on /
L. V. Lammert wrote: > At 08:31 PM 6/16/2005 +0100, Niall O'Higgins wrote: >>Controllers don't tend to like it. Sometimes with disk failure, the >>controller will fail too! > > The ASUS A7V880 runs just fine with one disk dead - infant mortality a few > months ago. > > Lee One example does not make it always so. Some people expect RAID (of either HW or SW kind) to keep them running through a disk failure... Some have more experience. Designing systems that work through failures is not trivial. The way devices fail in the real world is very different than the way you expect them to fail, and rarely can you get a device to fail while you are watching everything you need to to watch to fix a problem once discovered. If you do get a real-world failure which produces a problem, you try to fix it, but you will probably never know how well you fixed it, because it will never fail in exactly the same way again. If you try to manufacture defective drives (i.e., spike 'em with a powder-actuated nail gun while they are spinning), you will rack up a lot of money rapidly (at least for a volunteer project) (but it IS fun!). So, yes, I'm saying there are probably bugs in how HW failures are handled in OpenBSD...and probably most other OSs. It just isn't something you can test effectively, but only refine it over years of (bitter) experience. I've always told people RAID is part of a rapid-repair solution, not part of a "never goes down". It *may* not go down. Maybe, probably won't go down. But don't bet your career on it. Plan for the worst case, and things will always look better than expected. And you look like a genius. :) Nick.
Re: CCD on /
John Wright wrote: With the new raid autoconfiguration setup this is not the case. Not quite. I have a "home" server where the three large drives are on the P-ATA buses, but the boot/root drive is on S-ATA. I have to hack the kernel config to make sure that the root drive is always before (i.e. wd0) the P-ATA drives. Otherwise, if I left root as wd3 (which is fine at first) if one the RAIDed drives fails, boot fails as wd3 is no longer wd3, but wd2... It's not always the RAID config that gets you. Peter
ccd on active disks?
Hi misc Is anybody aware of a document that describes how to ccd all slices (including /) after installation? I've installed 3.8 generic using just one of two identical disks. Now I need to mirror that disk onto the other one. I copied the disklabel from the active disk over. I can of course create ccd's for each slice, but this means newfs and data loss, and won't work because I can't umount the root. Now I assume that my approach was intrinsically bound to fail or just dumb wrong. I'm sorry if this is covered somewhere public - I found references to a FAQ on MARC, but was just unable to find my question covered in the related documentation. One thing I came up with was to boot from install media and do it from the shell there, then newfs and reinstall on the ccd devices. But I'm really not quite sure. The goal is to bring the box back online as fast as possible after a disk crash and replacement, and to be then able to rebuild the raidset online (dd is fine). thx /markus
Re: ccd on active disks?
Markus Wernig wrote: Hi misc Is anybody aware of a document that describes how to ccd all slices (including /) after installation? I've installed 3.8 generic using just one of two identical disks. Now I need to mirror that disk onto the other one. I copied the disklabel from the active disk over. I can of course create ccd's for each slice, but this means newfs and data loss, and won't work because I can't umount the root. Now I assume that my approach was intrinsically bound to fail or just dumb wrong. I'm sorry if this is covered somewhere public - I found references to a FAQ on MARC, but was just unable to find my question covered in the related documentation. One thing I came up with was to boot from install media and do it from the shell there, then newfs and reinstall on the ccd devices. But I'm really not quite sure. The goal is to bring the box back online as fast as possible after a disk crash and replacement, and to be then able to rebuild the raidset online (dd is fine). thx /markus Hi Markus I've worked with 'ccd' on a number of systems, but only for the purpose of putting a number of slices into a single bigger one, most often /var. The easiest way of doint that is during install. If your want a raid mirror for the entire! disk(s) then have a look at bioctl. Yes, you can control how 'ccd' should write to the entries in your ccdconfig file, but it's still not a raid solution and 'ccd' on /root is not possible (not to my knowledge). /per [EMAIL PROTECTED]
Re: ccd on active disks?
On Thu, Nov 17, 2005 at 04:47:03PM +0100, Markus Wernig wrote: > Hi misc > > Is anybody aware of a document that describes how to ccd all slices > (including /) after installation? > I've installed 3.8 generic using just one of two identical disks. Now I > need to mirror that disk onto the other one. I copied the disklabel from > the active disk over. I can of course create ccd's for each slice, but > this means newfs and data loss, and won't work because I can't umount > the root. > Now I assume that my approach was intrinsically bound to fail or just > dumb wrong. I'm sorry if this is covered somewhere public - I found > references to a FAQ on MARC, but was just unable to find my question > covered in the related documentation. > One thing I came up with was to boot from install media and do it from > the shell there, then newfs and reinstall on the ccd devices. But I'm > really not quite sure. > > The goal is to bring the box back online as fast as possible after a > disk crash and replacement, and to be then able to rebuild the raidset > online (dd is fine). > > thx /markus Well, it's not ccd, but there are a couple of documents around that describe how to put root on a RAID-1. This requires a custom kernel, but works very well - for me at least. And if the point is disaster recovery, a RAID-1 is as good as ccd if you keep a spare custom kernel somewhere. I seem to recall that the kernel will not autoconfigure ccd. That means you can't put / on it, without monkeying around at least. However, you could put / on a RAID-1, and then use ccd for the rest (if truly paranoid, turn most of /etc, /opt, /dev and so on into symlinks to a ccd mount...) Joachim
Re: ccd on active disks?
Markus Wernig wrote: > Hi misc > > Is anybody aware of a document that describes how to ccd all slices > (including /) after installation? You are wanting to RETROFIT ccd mirroring on a disk that was set up without it? No. Not going to happen easily. ccd won't do root, though you could use the ALTROOT system to nightly duplicate root for you. How often do you change the data on the root partition? Hint: you don't need to mirror root. You do need to make sure your second drive has a bootable and pretty-current root, however. That's what ALTROOT does for you. You *might* be able to do a ccd setup on the second drive, copy your primary drive to it, boot off the new secondary drive, rebuild the first drive for ccd config, then duplicate. > I've installed 3.8 generic using just one of two identical disks. Now I > need to mirror that disk onto the other one. I copied the disklabel from > the active disk over. I can of course create ccd's for each slice, but > this means newfs and data loss, and won't work because I can't umount > the root. oh, you will be unmounting the entire system. > Now I assume that my approach was intrinsically bound to fail or just > dumb wrong. I'm sorry if this is covered somewhere public - I found > references to a FAQ on MARC, but was just unable to find my question > covered in the related documentation. > One thing I came up with was to boot from install media and do it from > the shell there, then newfs and reinstall on the ccd devices. But I'm > really not quite sure. you will need to /practice/ on a non-production system. I don't care what RAID solution you use. This is a must. > The goal is to bring the box back online as fast as possible after a > disk crash and replacement, and to be then able to rebuild the raidset > online (dd is fine). rebuilding with dd is done OFF LINE. You can't duplicate a live file system. You also want to make sure your ccd'd partitions are as small as you can get away with, yes, with most of your disk unused. That's to get rebuild time down as low as possible. Are you working with a system with so much dynamic data that you can't use a nightly (or weekly) dump/restore to a second drive? This can be easily done after initial install. You can't easily retrofit S/W RAID into an already configured system. Nick.
