Re: [RDD] Redundant Hard Drive/Backup
On Sunday 16 February 2014 12:22:49 am Stan Fotinos wrote: > You could > buy a sata expansion card to add more ports I would guess, never tried > this... Has anyone done this bofore? Yep. Works fine. -- Cowboy http://cowboy.cwf1.com Call on God, but row away from the rocks. -- Indian proverb ___ Rivendell-dev mailing list Rivendell-dev@lists.rivendellaudio.org http://caspian.paravelsystems.com/mailman/listinfo/rivendell-dev
Re: [RDD] Redundant Hard Drive/Backup
I purposely buy PCI-Express HBAs (host-bus adapters) instead of full blown raid cards for use with software raid systems. They're a lot cheaper than reliable hardware raid cards ($100s vs $1000s) and are a lot more reliable than any cheap fake-raid (raid-in-driver) card. I've even "downgraded" the firmware on some full blown raid cards to make them basic HBAs in order to make sure none of their proprietary RAID stack would get in the way of a clean software raid setup. I normally buy 8-port SATA HBAs and am currently running several 20TB arrays using Linux mdraid, 8-port LSI HBAs, 4TB drives, and RAID6. So yes, people can and do buy SATA expansion cards (HBAs) for use with software raid. On Sat, Feb 15, 2014 at 10:22 PM, Stan Fotinos wrote: > If you need to connect lots of drives, software raid can use the maximum > number of drives ports that your motherboard has ie 6 drives. You could buy > a sata expansion card to add more ports I would guess, never tried this... > Has anyone done this bofore? > > With one hardware raid card could plugin 24 drives :-) > > Stan > > > On 15/02/14 9:26 AM, Cowboy wrote: > >> On Friday 14 February 2014 05:32:08 pm Jim Stewart wrote: >> >>> 3) Linux RAID also seem less picky about choice of hard drives as >>> you can mix and match (although typically not the greatest idea for >>> performance reasons), and all is fine. >>> >> That's because Linux software RAID is partition based, not device based. >> You could build a RAID array on a single disk, though there would be >> no advantage, and several disadvantages to doing so. >> >> > ___ > Rivendell-dev mailing list > Rivendell-dev@lists.rivendellaudio.org > http://caspian.paravelsystems.com/mailman/listinfo/rivendell-dev > ___ Rivendell-dev mailing list Rivendell-dev@lists.rivendellaudio.org http://caspian.paravelsystems.com/mailman/listinfo/rivendell-dev
Re: [RDD] Redundant Hard Drive/Backup
If you need to connect lots of drives, software raid can use the maximum number of drives ports that your motherboard has ie 6 drives. You could buy a sata expansion card to add more ports I would guess, never tried this... Has anyone done this bofore? With one hardware raid card could plugin 24 drives :-) Stan On 15/02/14 9:26 AM, Cowboy wrote: On Friday 14 February 2014 05:32:08 pm Jim Stewart wrote: 3) Linux RAID also seem less picky about choice of hard drives as you can mix and match (although typically not the greatest idea for performance reasons), and all is fine. That's because Linux software RAID is partition based, not device based. You could build a RAID array on a single disk, though there would be no advantage, and several disadvantages to doing so. ___ Rivendell-dev mailing list Rivendell-dev@lists.rivendellaudio.org http://caspian.paravelsystems.com/mailman/listinfo/rivendell-dev
Re: [RDD] Redundant Hard Drive/Backup
On Friday 14 February 2014 05:32:08 pm Jim Stewart wrote: > 3) Linux RAID also seem less picky about choice of hard drives as you > can mix and match (although typically not the greatest idea for performance > reasons), and all is fine. That's because Linux software RAID is partition based, not device based. You could build a RAID array on a single disk, though there would be no advantage, and several disadvantages to doing so. -- Cowboy http://cowboy.cwf1.com It is by the fortune of God that, in this country, we have three benefits: freedom of speech, freedom of thought, and the wisdom never to use either. -- Mark Twain ___ Rivendell-dev mailing list Rivendell-dev@lists.rivendellaudio.org http://caspian.paravelsystems.com/mailman/listinfo/rivendell-dev
Re: [RDD] Redundant Hard Drive/Backup
On Friday 14 February 2014 05:33:51 pm nathan lawson wrote: > Just be aware that Software RAID has its settings saved in the software of > the OS so it can be a right mare to recover from. Where did you here THAT ?? There's a partition type "Linux RAID" for a reason ! Depending on which you use, the recovery machine doesn't even have to be RAID capable to use the disk as if RAID didn't even exist ! -- Cowboy http://cowboy.cwf1.com It is by the fortune of God that, in this country, we have three benefits: freedom of speech, freedom of thought, and the wisdom never to use either. -- Mark Twain ___ Rivendell-dev mailing list Rivendell-dev@lists.rivendellaudio.org http://caspian.paravelsystems.com/mailman/listinfo/rivendell-dev
Re: [RDD] Redundant Hard Drive/Backup
You might consider Steve Gibson's SpinRite 6.0 by Gibson Research Corporation (grc.com). Many of his customers do just that - they run SpinRite on NEW drives (even though it's often used as a preventive maintenance or disaster recovery utility). It puts them through their paces and analyzes them thoroughly. I believe it does mark marginal bad sectors as "bad" if it cannot revitalize them. I actually used SpinRite (I think v1.0 !) regularly on my first hard drive (30MB RLL!). That product has been around for about 30 years, and is legitimate. He's re-writing it currently from the ground up, and if you buy the current version (6.0), upgrades to 6.x and 7.0 (the new re-written one) will be free. I think the price is around $89.00. He also has a great weekly podcast called Security Now! you might check out. I have no formal affiliation with grc.com - this is just a personal recommendation. Cheers, Rick Quendun KMUZ Engineering From: nathan lawson To: Jim Stewart Cc: "rivendell-dev@lists.rivendellaudio.org" Sent: Friday, February 14, 2014 2:33 PM Subject: Re: [RDD] Redundant Hard Drive/Backup Just be aware that Software RAID has its settings saved in the software of the OS so it can be a right mare to recover from. Thats when i decided to start looking at even the lower end hardware cards... Regards On Fri, Feb 14, 2014 at 10:32 PM, Jim Stewart wrote: Here is what I will add to this RAID discussion: > >1) The best description of the difference between hardware and software >RAID is “what CPU is doing the RAID”. A true hardware RAID doesn’t tax the >host system for RAID functions. That said on many workloads, (like I’d expect >most Rivendell ones) there is likely plenty of spare CPU cycles to do RAID >while the CPU is likely just waiting for I/O anyway. Video Rendering is >likely a different story! >2) People do get a false sense of security with RAID. As previously >mentioned, it does not protect you from corruption, accidental deletions, etc. > More than that people often don’t consider how they are going to have to >deal with an actual RAID system failure. Consider the following: >a. I’ve seen I many times, someone has a fancy, high-priced hardware >RAID system on a mission critical system so that they can sleep nights feeling >pretty protected. Suddenly the RAID hardware goes down! Did they think about >having a space RAID box laying around? No! They have to get a new one flown >in at great expense only to be out priced by the expense of the actual down >time! >b. Okay so you have one of those motherboard BIOS based software RAID >setups (that a lot of people *think* is hardware RAID), in this case you >typically get the worse parts of software and hardware RAID in this situation. > Not only are you still stealing main CPU cycles, but once again now the >motherboard goes down and you have to find another one that does that system’s >way of doing RAID! >c. Now consider Linux software RAID. You can have all the hardware >failures you want and simply boot your Linux + RAID set up on new hardware and >you are up and running again! The only drawback here is your stuck with >running Linux (LOL) to operate your RAID. >3) Linux RAID also seem less picky about choice of hard drives as you can >mix and match (although typically not the greatest idea for performance >reasons), and all is fine. Also I don’t know about those BIOS RAID solutions, >but if you have hot-swapable drives, you shouldn’t have to shut down your >system to replace and rebuild drives. Granted most good hardware RAID systems >give you this too. > >I’ve been subject to another advantage of RAID: I’ve recently had lots of >trouble with modern hard drives that have “not-quite-defective” sectors. This >has been a real pain for me as the whole system stalls out as the hard drive >struggles to read data in a “not officially bad sector”, which it eventually >does, but only after a system slowdown. I wish someone would write a good >disk tester that as real short time-outs so to mark these marginal areas bad >and be done with it! Anyway, with RAID mirroring, it seems like the system >runs just fine (as long as you are reading, not writing) as any bad spots on >one drive are read instead by the other one in the mirror set. Yea I know, >why am I messing with bad drives? The truth is I can’t seem to find any that >don’t do this these days, I think I’ve tried all the (few remaining) hard >drive makes/models there are. I’ve been told that if I go with some sort of >“high-end” drives like SAS interface ones, that the QC is higher on them and I probably won’t have the problem. It would be too bad if this is what it
Re: [RDD] Redundant Hard Drive/Backup
On Fri, Feb 14, 2014 at 3:33 PM, nathan lawson wrote: > Just be aware that Software RAID has its settings saved in the software of > the OS so it can be a right mare to recover from. Thats when i decided to > start looking at even the lower end hardware cards... That's not true. Linux md raid saves all it's core configuration and metadata in the superblock at the start of each partition. You can literally grab all of the drives from one Linux mdraid array, slap then in another Linux box that has never seen them before, and they'll show up just fine as a cohesive array (possibly requiring you to tell the OS to scan for them or to reboot first). Good luck doing that with most hardware raid solutions... As long as you have the spanning set of drives from an md array and a Linux box to connect them to, you can mount the array and access your data. ___ Rivendell-dev mailing list Rivendell-dev@lists.rivendellaudio.org http://caspian.paravelsystems.com/mailman/listinfo/rivendell-dev
Re: [RDD] Redundant Hard Drive/Backup
Just be aware that Software RAID has its settings saved in the software of the OS so it can be a right mare to recover from. Thats when i decided to start looking at even the lower end hardware cards... Regards On Fri, Feb 14, 2014 at 10:32 PM, Jim Stewart wrote: > Here is what I will add to this RAID discussion: > > > > 1) The best description of the difference between hardware and > software RAID is "what CPU is doing the RAID". A true hardware RAID > doesn't tax the host system for RAID functions. That said on many > workloads, (like I'd expect most Rivendell ones) there is likely plenty of > spare CPU cycles to do RAID while the CPU is likely just waiting for I/O > anyway. Video Rendering is likely a different story! > > 2) People do get a false sense of security with RAID. As previously > mentioned, it does not protect you from corruption, accidental deletions, > etc. More than that people often don't consider how they are going to > have to deal with an actual RAID system failure. Consider the following: > > a. I've seen I many times, someone has a fancy, high-priced > hardware RAID system on a mission critical system so that they can sleep > nights feeling pretty protected. Suddenly the RAID hardware goes down! > Did they think about having a space RAID box laying around? No! They have > to get a new one flown in at great expense only to be out priced by the > expense of the actual down time! > > b. Okay so you have one of those motherboard BIOS based software > RAID setups (that a lot of people **think** is hardware RAID), in this > case you typically get the worse parts of software and hardware RAID in > this situation. Not only are you still stealing main CPU cycles, but once > again now the motherboard goes down and you have to find another one that > does that system's way of doing RAID! > > c. Now consider Linux software RAID. You can have all the hardware > failures you want and simply boot your Linux + RAID set up on new hardware > and you are up and running again! The only drawback here is your stuck > with running Linux (LOL) to operate your RAID. > > 3) Linux RAID also seem less picky about choice of hard drives as > you can mix and match (although typically not the greatest idea for > performance reasons), and all is fine. Also I don't know about those BIOS > RAID solutions, but if you have hot-swapable drives, you shouldn't have to > shut down your system to replace and rebuild drives. Granted most good > hardware RAID systems give you this too. > > > > I've been subject to another advantage of RAID: I've recently had lots of > trouble with modern hard drives that have "not-quite-defective" sectors. > This has been a real pain for me as the whole system stalls out as the hard > drive struggles to read data in a "not officially bad sector", which it > eventually does, but only after a system slowdown. I wish someone would > write a good disk tester that as real short time-outs so to mark these > marginal areas bad and be done with it! Anyway, with RAID mirroring, it > seems like the system runs just fine (as long as you are reading, not > writing) as any bad spots on one drive are read instead by the other one in > the mirror set. Yea I know, why am I messing with bad drives? The truth > is I can't seem to find any that don't do this these days, I think I've > tried all the (few remaining) hard drive makes/models there are. I've been > told that if I go with some sort of "high-end" drives like SAS interface > ones, that the QC is higher on them and I probably won't have the problem. > It would be too bad if this is what it takes. > > ___ > Rivendell-dev mailing list > Rivendell-dev@lists.rivendellaudio.org > http://caspian.paravelsystems.com/mailman/listinfo/rivendell-dev > > -- Nathan Lawson ___ Rivendell-dev mailing list Rivendell-dev@lists.rivendellaudio.org http://caspian.paravelsystems.com/mailman/listinfo/rivendell-dev
Re: [RDD] Redundant Hard Drive/Backup
Here is what I will add to this RAID discussion: 1) The best description of the difference between hardware and software RAID is "what CPU is doing the RAID". A true hardware RAID doesn't tax the host system for RAID functions. That said on many workloads, (like I'd expect most Rivendell ones) there is likely plenty of spare CPU cycles to do RAID while the CPU is likely just waiting for I/O anyway. Video Rendering is likely a different story! 2) People do get a false sense of security with RAID. As previously mentioned, it does not protect you from corruption, accidental deletions, etc. More than that people often don't consider how they are going to have to deal with an actual RAID system failure. Consider the following: a. I've seen I many times, someone has a fancy, high-priced hardware RAID system on a mission critical system so that they can sleep nights feeling pretty protected. Suddenly the RAID hardware goes down! Did they think about having a space RAID box laying around? No! They have to get a new one flown in at great expense only to be out priced by the expense of the actual down time! b. Okay so you have one of those motherboard BIOS based software RAID setups (that a lot of people *think* is hardware RAID), in this case you typically get the worse parts of software and hardware RAID in this situation. Not only are you still stealing main CPU cycles, but once again now the motherboard goes down and you have to find another one that does that system's way of doing RAID! c. Now consider Linux software RAID. You can have all the hardware failures you want and simply boot your Linux + RAID set up on new hardware and you are up and running again! The only drawback here is your stuck with running Linux (LOL) to operate your RAID. 3) Linux RAID also seem less picky about choice of hard drives as you can mix and match (although typically not the greatest idea for performance reasons), and all is fine. Also I don't know about those BIOS RAID solutions, but if you have hot-swapable drives, you shouldn't have to shut down your system to replace and rebuild drives. Granted most good hardware RAID systems give you this too. I've been subject to another advantage of RAID: I've recently had lots of trouble with modern hard drives that have "not-quite-defective" sectors. This has been a real pain for me as the whole system stalls out as the hard drive struggles to read data in a "not officially bad sector", which it eventually does, but only after a system slowdown. I wish someone would write a good disk tester that as real short time-outs so to mark these marginal areas bad and be done with it! Anyway, with RAID mirroring, it seems like the system runs just fine (as long as you are reading, not writing) as any bad spots on one drive are read instead by the other one in the mirror set. Yea I know, why am I messing with bad drives? The truth is I can't seem to find any that don't do this these days, I think I've tried all the (few remaining) hard drive makes/models there are. I've been told that if I go with some sort of "high-end" drives like SAS interface ones, that the QC is higher on them and I probably won't have the problem. It would be too bad if this is what it takes. ___ Rivendell-dev mailing list Rivendell-dev@lists.rivendellaudio.org http://caspian.paravelsystems.com/mailman/listinfo/rivendell-dev
Re: [RDD] Redundant Hard Drive/Backup
On Feb 14, 2014, at 13:04 56, Lorne Tyndale wrote: > For now I'd recommend sticking with tried and true RAID systems with > otherwise proven file systems. FWIW, we have one customer that has been running ZFS on the audio store and has been having problems with latency and audio starvation. While I don’t have a ‘smoking gun’ that ZFS is the culprit, everything else in the stack at that site is stock Broadcast Appliance. Cheers! |-| | Frederick F. Gleason, Jr. | Chief Developer | | | Paravel Systems | |-| | There are two ways of constructing a software design. One is to make | | it so simple that there are obviously no deficiencies; the other is| | to make it so complicated that there are no obvious deficiencies. | | The first method is far more difficult.| | --C.A.R Hoare | |-| ___ Rivendell-dev mailing list Rivendell-dev@lists.rivendellaudio.org http://caspian.paravelsystems.com/mailman/listinfo/rivendell-dev
Re: [RDD] Redundant Hard Drive/Backup
Hi, While BTRFS does look very promising, and does appear to offer features which go beyond other file systems out there, I'd question whether it would be worth the risk of putting into a mission critical system. Just the fact that the main wiki has a "stability status" entry suggests to me that while it is likely worthwhile to test out to see its capabilities, I'd be very hesitant to put it on anything other then a test system. For now I'd recommend sticking with tried and true RAID systems with otherwise proven file systems. Lorne Tyndale > > > On Fri, Feb 14, 2014 at 5:10 AM, Cowboy wrote: > > > Depends on the objective. > > If the objective is throughput, I just don't see how a *file system* > > can outrun parallel simultaneous read across multiple devices ? > > OTOH, if the objective is data integrity, then it becomes debatable. > > If the objective is maximizing disk space while retaining data integrity, > > RAID-5 is hard to beat. > > > > btrfs is perfectly capable of reading from (or writing to) multiple devices > simultaneously. That's a core part of its raid-like multi-device support. > In fact, it has many speed advantages over a traditional data-agnostic raid > system: ability to flag and prioritize 'hot' data, ability to ignore > unwritten parts of the disk, support for on-the-fly compression to speed > effective disk i/o, etc. btrfs parity support (e.g. raid5 functionality) is > still not completely baked and should not be used in a critical production > system, but early tests show it outperforming a tradition md raid5 setup > both in terms of speed and in terms of data redundancy (since it can detect > and correct individual file corruption errors in addition to full disk > failures). The same can be said for the btrfs raid1 and raid10 > implementations (which are stable and safely usable today). And it's only > getting faster as development continues. > > See: > > https://btrfs.wiki.kernel.org/index.php/Multi-device_Benchmarks > http://theaveragegeek.blogspot.com/2013/02/unofficial-btrfs-raid5-benchmark.html___ > Rivendell-dev mailing list > Rivendell-dev@lists.rivendellaudio.org > http://caspian.paravelsystems.com/mailman/listinfo/rivendell-dev ___ Rivendell-dev mailing list Rivendell-dev@lists.rivendellaudio.org http://caspian.paravelsystems.com/mailman/listinfo/rivendell-dev
Re: [RDD] Redundant Hard Drive/Backup
On Fri, Feb 14, 2014 at 5:10 AM, Cowboy wrote: > Depends on the objective. > If the objective is throughput, I just don't see how a *file system* > can outrun parallel simultaneous read across multiple devices ? > OTOH, if the objective is data integrity, then it becomes debatable. > If the objective is maximizing disk space while retaining data integrity, > RAID-5 is hard to beat. > btrfs is perfectly capable of reading from (or writing to) multiple devices simultaneously. That's a core part of its raid-like multi-device support. In fact, it has many speed advantages over a traditional data-agnostic raid system: ability to flag and prioritize 'hot' data, ability to ignore unwritten parts of the disk, support for on-the-fly compression to speed effective disk i/o, etc. btrfs parity support (e.g. raid5 functionality) is still not completely baked and should not be used in a critical production system, but early tests show it outperforming a tradition md raid5 setup both in terms of speed and in terms of data redundancy (since it can detect and correct individual file corruption errors in addition to full disk failures). The same can be said for the btrfs raid1 and raid10 implementations (which are stable and safely usable today). And it's only getting faster as development continues. See: https://btrfs.wiki.kernel.org/index.php/Multi-device_Benchmarks http://theaveragegeek.blogspot.com/2013/02/unofficial-btrfs-raid5-benchmark.html ___ Rivendell-dev mailing list Rivendell-dev@lists.rivendellaudio.org http://caspian.paravelsystems.com/mailman/listinfo/rivendell-dev
Re: [RDD] Redundant Hard Drive/Backup
On Friday 14 February 2014 12:55:29 am Andy Sayler wrote: > As an aside, it's worth noting that traditional raid solutions are starting > to go out of favor and are being replaced by full-stack next-gen file > system like btrfs or zfs. I dunno Depends on the objective. If the objective is throughput, I just don't see how a *file system* can outrun parallel simultaneous read across multiple devices ? OTOH, if the objective is data integrity, then it becomes debatable. If the objective is maximizing disk space while retaining data integrity, RAID-5 is hard to beat. Really, it comes down to the house policy. Not what, but *why* a particular course is chosen. -- Cowboy http://cowboy.cwf1.com It is by the fortune of God that, in this country, we have three benefits: freedom of speech, freedom of thought, and the wisdom never to use either. -- Mark Twain ___ Rivendell-dev mailing list Rivendell-dev@lists.rivendellaudio.org http://caspian.paravelsystems.com/mailman/listinfo/rivendell-dev
Re: [RDD] Redundant Hard Drive/Backup
Hi Alan, When you say software raid1, are you referring to Linux software raid (e.g. mdraid)? Linux software raid is very fast on modern hardware, and often performs as well if not faster than any normal hardware raid solution. It's also a lot easier to recover from system failures with it since you can stick your hard drives in any Linux machine and access the RAID data, without being tied to a specific model of raid controller, proprietary drivers, etc. The general consensus in modern data-center and server applications running Linux is that md software raid is the way to go, and is generally preferable to any proprietary hardware raid system. So yes, it should be more than adequate for your needs from a speed and redundancy perspective. That said, remember the raid is not backup. It provides some limited protection against disk drive failure, but it's never a replacement for a backup system. It will not protect you if you accidentally delete a bunch of files, or if you file system gets corrupted. Thus, even when using a raid setup, it's still important to keep regular separate backups on a unrelated disk or array (e.g. an external hard drive or cloud server). Those will protect you if you ever need to restore files due to user error or software malfunction, for which raid will not help. As an aside, it's worth noting that traditional raid solutions are starting to go out of favor and are being replaced by full-stack next-gen file system like btrfs or zfs. These systems include raid-like functions as part of the file system, allowing them to do things like recover corrupted files by tracking checksums (something traditional raid can not do since it's completely agnostic to the data it's storing). These systems still don't exempt you from making regular backups, but they provide a lot of nice additions to the traditional raid paradigm. Cheers, Andy On Thu, Feb 13, 2014 at 9:04 PM, Alan Smith wrote: > I'm digging back up my old post (below), to ask a pretty easy question: > > How about using software raid1 instead of my convoluted efforts below? Is > performance okay enough for Rivendell to operate seamlessly with it? > > The longer version: > > Almost 10 years ago I build our fileserver at work on RHEL4 with a > proprietary raid (fake raid) card for the data, but the boot drive was > separate. It finally crapped out last week (primary HD failure). > > I am still VERY NEW to linux, so for a replacement I set my sights on > CentOS 6.4 (no coincidence the Rivendell Appliance V2 uses the same OS 8) > ). Anyway, I'm slowly figuring things out again, and today got software > Raid1 working on a pair of 2TB drives, including booting! > > I'm really happy with it, and this would be PERFECT as this is *almost* > how our current systems are set up (read below-they aren't true Raid, but > probably as close as you can get to a "software" raid for MS-DOS). > > Thanks everyone, > > -Alan > > On 1/8/2014 12:33 PM, Alan Smith wrote: > >> It may be a tad early yet, but I am about to the point in my comfort with >> Rivendell to start thinking about backups. >> >> I know this may not be the most common way, but we have our reasons, and >> I'd like to duplicate this behavior if I can: >> >> Our current automation system [MS-DOS based] contains two non-raid, but >> mirrored hard drives. On a schedule that we specify, it copies the new >> files, deletes the obsolete ones, etc. >> >> Is there a way I can do this with Rivendell? Again, I realize most >> people are probably using a central server with a redundant server to house >> all station audio, clocks, etc, but I am comfortable with the way we are >> set up now. Besides we have quite a few stations spread out, so a central >> file store wouldn't work anyway in most of our cases. >> >> With my very limited knowledge of linux, I thought perhaps running dd >> with chron might be the ticket, but coming from a Windows guy, I don't know >> how linux behaves when attempting to copy files in use. >> >> Thanks for any suggestions. >> >> Oh, and all my Rivendell installs are/will be based on the Paravel >> Appliances. I have one V1 appliance in the field now, and currently >> working on my first V2. >> >> -Alan >> >> PS We actually had a hard drive die this past Saturday on one of our DOS >> stations. I just simply powered down, unlocked the bad drive from its >> removable cage, powered back up, and we were back in business. >> ___ >> Rivendell-dev mailing list >> Rivendell-dev@lists.rivendellaudio.org >> http://caspian.paravelsystems.com/mailman/listinfo/rivendell-dev >> >> >> >> - >> No virus found in this message. >> Checked by AVG - www.avg.com >> Version: 2013.0.3462 / Virus Database: 3658/6985 - Release Date: 01/08/14 >> >> >> > ___ > Rivendell-dev mailing list > Rivendell-dev@lists.rivendellaudio.org > http://caspian.paravelsystems.com/mailman/listinfo/rivendell-dev >
Re: [RDD] Redundant Hard Drive/Backup
I'm digging back up my old post (below), to ask a pretty easy question: How about using software raid1 instead of my convoluted efforts below? Is performance okay enough for Rivendell to operate seamlessly with it? The longer version: Almost 10 years ago I build our fileserver at work on RHEL4 with a proprietary raid (fake raid) card for the data, but the boot drive was separate. It finally crapped out last week (primary HD failure). I am still VERY NEW to linux, so for a replacement I set my sights on CentOS 6.4 (no coincidence the Rivendell Appliance V2 uses the same OS 8) ). Anyway, I'm slowly figuring things out again, and today got software Raid1 working on a pair of 2TB drives, including booting! I'm really happy with it, and this would be PERFECT as this is *almost* how our current systems are set up (read below-they aren't true Raid, but probably as close as you can get to a "software" raid for MS-DOS). Thanks everyone, -Alan On 1/8/2014 12:33 PM, Alan Smith wrote: It may be a tad early yet, but I am about to the point in my comfort with Rivendell to start thinking about backups. I know this may not be the most common way, but we have our reasons, and I'd like to duplicate this behavior if I can: Our current automation system [MS-DOS based] contains two non-raid, but mirrored hard drives. On a schedule that we specify, it copies the new files, deletes the obsolete ones, etc. Is there a way I can do this with Rivendell? Again, I realize most people are probably using a central server with a redundant server to house all station audio, clocks, etc, but I am comfortable with the way we are set up now. Besides we have quite a few stations spread out, so a central file store wouldn't work anyway in most of our cases. With my very limited knowledge of linux, I thought perhaps running dd with chron might be the ticket, but coming from a Windows guy, I don't know how linux behaves when attempting to copy files in use. Thanks for any suggestions. Oh, and all my Rivendell installs are/will be based on the Paravel Appliances. I have one V1 appliance in the field now, and currently working on my first V2. -Alan PS We actually had a hard drive die this past Saturday on one of our DOS stations. I just simply powered down, unlocked the bad drive from its removable cage, powered back up, and we were back in business. ___ Rivendell-dev mailing list Rivendell-dev@lists.rivendellaudio.org http://caspian.paravelsystems.com/mailman/listinfo/rivendell-dev - No virus found in this message. Checked by AVG - www.avg.com Version: 2013.0.3462 / Virus Database: 3658/6985 - Release Date: 01/08/14 ___ Rivendell-dev mailing list Rivendell-dev@lists.rivendellaudio.org http://caspian.paravelsystems.com/mailman/listinfo/rivendell-dev
Re: [RDD] Redundant Hard Drive/Backup
On Wednesday 08 January 2014 06:42:36 pm Alan Smith wrote: > So now its down to rsync via chron, or software raid... > Now that I understand, I'd recommend you do what I do, or something along these lines.. Software RAID-1. with a twist. Each physical disk has 3 partitions. 1 swap, 1 very basic minimal system, 1 everything else. This way, disk reads are faster, ( it's striped in the RAID driver ) but disk writes are slower. ( must write the same data twice ) Swap is striped, so swap is faster. ( if you're short on RAM ) Either disk can function complete as a non-RAID stand alone device, and/or can be accessed directly without a RAID driver in the kernel, in the event it becomes necessary. ( some day, some way, it will ) The basic system is one very small RAID-1 partition. ( about 200 MB ) /boot /bin /sbin /etc /lib a number of mount points, and not much else. Things that essentially never change. That partition is also backed up on a bootable CD. Everything else is a RAID-1 partition that houses files that can, do, or might change with some frequency. That md partition is rsync'd to another machine as a backup. RAID IS **NOT** A BACKUP !! RAID will not help you if the motherboard fails, for instance. Periodically ( daily, weekly ) removing one of the physical disks, putting it on a shelf, and replacing it with another good disk, does leave you with a backup on the shelf. rsync can do this for you, but requires another machine on the network to house that backup. The kernel maintains mdstat in /proc which is the current health of the RAID md devices dynamically updated constantly. A cron job compares a recorded copy of mdstat to the dynamic /proc/mdstat file every 5 minutes or so. If they are not identicle, something has changed, and I want to know very quickly. The system starts screaming for attention, sending e-mails, flashing the screen, beeping the speaker but keeps on running off of the non-failed device. If the disks were purchased and installed at the same time, you've got about a week to deal with it. Disks manufactured at the same time, with the same run time on them, tend to fail within about 8 days of each other on average. Maybe longer, but I wouldn't. That way, I can replace the failed drive, re-boot and let the system rebuild the RAID with about 5 minutes total off-line down time. Rebuilding a 2 terabyte RAID takes hours, but the system is up and running while it happens, with a little planning. If there is a spare disk in the machine, the RAID driver can swap out the failed disk all by itself, but that's a little advanced. -- Cowboy http://cowboy.cwf1.com Never eat more than you can lift. -- Miss Piggy ___ Rivendell-dev mailing list Rivendell-dev@lists.rivendellaudio.org http://caspian.paravelsystems.com/mailman/listinfo/rivendell-dev
Re: [RDD] Redundant Hard Drive/Backup
Again, THANK EVERYONE. This list truly is a blessing. I would be so lost without all of your help. Got pulled off working on Rivendell for a bit, but the quick research I did suggested doing a dd of a live file system is NOT a good idea, and I don't know why I didn't think of that before. Someone asked above about our current setup-It isn't really a "mirror" in the sense of an EXACT copy, but it is mirrored in the sense that on a schedule in compares all the files on the main drive and copies/deletes files as necessary on the backup drive to make sure they are a duplicate in the sense of files. I don't need an exact copy of the drive. Just from a Rivendell/Linux newbie standpoint, it would be a lot easier if I could just pull a failed drive and turn it back on and be up and running, but so long as I have the audio library, database, and system specific setups (events, clocks, grids, etc), then I could at least quickly throw together another machine and go from there. Although software raid sounds promising as well. So now its down to rsync via chron, or software raid... Thanks again everyone! -Alan ___ Rivendell-dev mailing list Rivendell-dev@lists.rivendellaudio.org http://caspian.paravelsystems.com/mailman/listinfo/rivendell-dev
Re: [RDD] Redundant Hard Drive/Backup
On Wednesday 08 January 2014 03:17:42 pm Cowboy wrote: > dd is not a good idea. > dd can destroy your data in one bit. Just to elaborate a little... dd is (D)evice to (D)evice. Doesn't matter what that device is, or isn't. Want to copy your monitor display over your partition tables, pixel for pixel ? dd is your tool. Want to write your boot block to your modem ? Again, dd is the tool. Want to copy an entire disk, including boot block, partition tables, geometry representations, and data, bit for bit into a file ? Again, dd is your tool. Yes, Virginia, you can create a bootable text file, and dd will do it. Want to copy one file to another file ? dd is not the best tool in the box for that. cp is quick, and simple, but limited. rsync can do everything cp can do, and much more, but it can't do the kinds of things dd can do. Can dd copy files ? Yes, it can, if you give it the correct command options and in the correct order. Whatever command options you don't give it, it will assume some defaults, like byte offset, or block size, what to do when an error is encountered. ( which is stop dead in it's tracks. Do not back out, and make no attempt to recover. Just stop ) If you tell it to copy a file, will it check first to make sure that the file you're copying actually is a file, or is locked by some other process ? NO ! It will blindly copy whatever it thinks you've told it, and that's where it can get you into trouble. A file that is currently being updated, dd will likely create trash at the target, where neither cp nor rsync will do that. -- Cowboy http://cowboy.cwf1.com You're never too old to become younger. -- Mae West ___ Rivendell-dev mailing list Rivendell-dev@lists.rivendellaudio.org http://caspian.paravelsystems.com/mailman/listinfo/rivendell-dev
Re: [RDD] Redundant Hard Drive/Backup
On Wednesday 08 January 2014 01:33:33 pm Alan Smith wrote: > It may be a tad early yet, but I am about to the point in my comfort > with Rivendell to start thinking about backups. > > I know this may not be the most common way, but we have our reasons, and > I'd like to duplicate this behavior if I can: I'm certain you can. It's a matter of how. > Our current automation system [MS-DOS based] contains two non-raid, but > mirrored hard drives. On a schedule that we specify, it copies the new > files, deletes the obsolete ones, etc. OK, Mirror, means that the disk is a second verbatim copy of another disk. That's RAID-1. On schedule, it copies what to where ? If I'm understanding correctly, your "mirror" isn't really a mirror, but a backup copy of the working disk, periodically updated ? If so, then cron and rsync are your best options. rsync can intelligently manage updating, active files, etc. It can easily work as a fairly sophisticated intelligent cp, machine to another machine, or on the same machine. cron can run it every minute, to once and only once, or anything in between. Probably, I'd run rsync every 5 minutes or so. > With my very limited knowledge of linux, I thought perhaps running dd > with chron might be the ticket, but coming from a Windows guy, I don't > know how linux behaves when attempting to copy files in use. dd is not a good idea. dd can destroy your data in one bit. rsync won't/can't do that. Any *nix copies the file as it is when the copy program gets the file handle. If something else subsequently gets a later file handle, then the first one isn't released or updated until the first one is released. Not before that moment is the file contents lost or updated, as far as that process is concerned. Any other later process would see the newer updated file. In other words, Windows creates a train wreck. Linux ( more correctly, the file system driver ) handles it intelligently, sequentially, and in proper order as one would expect. -- Cowboy http://cowboy.cwf1.com You're never too old to become younger. -- Mae West ___ Rivendell-dev mailing list Rivendell-dev@lists.rivendellaudio.org http://caspian.paravelsystems.com/mailman/listinfo/rivendell-dev
Re: [RDD] Redundant Hard Drive/Backup
Hi, Along with rsync you could create a shell script to dump the MySQL database from system and save it to the secondary disc. In case you need it you'll have to import it only. And if the replicated disc is also the system disc, make sure to have grub properly installed, so system can boot from this disc too. Regards Atenciosamente, *Fernando Della Torre* Tecnologia da Informação (: +55 16 98137-1240 (: +55 16 99137-2886 *: *f...@vdit.com.br * V.D.I.T. Soluções em Virtualização A utilização deste e-mail não implica em autorização ou outorga de poderes para seu usuário praticar qualquer ato em nome das empresas citadas, cuja representação considera-se válida se praticada exclusivamente por representante legal ou procurador devidamente constituído, na forma estabelecida em seu respectivo estatuto ou contrato social 2014/1/8 Nathan Steele > Rsync > > I do the same thing on a samba server. > > Google it for the options you want to use. > > > -- > > On January 8, 2014 1:33:41 PM Alan Smith wrote: > > It may be a tad early yet, but I am about to the point in my comfort with >> Rivendell to start thinking about backups. >> >> I know this may not be the most common way, but we have our reasons, and >> I'd like to duplicate this behavior if I can: >> >> Our current automation system [MS-DOS based] contains two non-raid, but >> mirrored hard drives. On a schedule that we specify, it copies the new >> files, deletes the obsolete ones, etc. >> >> Is there a way I can do this with Rivendell? Again, I realize most >> people are probably using a central server with a redundant server to house >> all station audio, clocks, etc, but I am comfortable with the way we are >> set up now. Besides we have quite a few stations spread out, so a central >> file store wouldn't work anyway in most of our cases. >> >> With my very limited knowledge of linux, I thought perhaps running dd >> with chron might be the ticket, but coming from a Windows guy, I don't know >> how linux behaves when attempting to copy files in use. >> >> Thanks for any suggestions. >> >> Oh, and all my Rivendell installs are/will be based on the Paravel >> Appliances. I have one V1 appliance in the field now, and currently >> working on my first V2. >> >> -Alan >> >> PS We actually had a hard drive die this past Saturday on one of our DOS >> stations. I just simply powered down, unlocked the bad drive from its >> removable cage, powered back up, and we were back in business. >> ___ >> Rivendell-dev mailing list >> Rivendell-dev@lists.rivendellaudio.org >> http://caspian.paravelsystems.com/mailman/listinfo/rivendell-dev >> >> > > ___ > Rivendell-dev mailing list > Rivendell-dev@lists.rivendellaudio.org > http://caspian.paravelsystems.com/mailman/listinfo/rivendell-dev > ___ Rivendell-dev mailing list Rivendell-dev@lists.rivendellaudio.org http://caspian.paravelsystems.com/mailman/listinfo/rivendell-dev
Re: [RDD] Redundant Hard Drive/Backup
Rsync I do the same thing on a samba server. Google it for the options you want to use. -- On January 8, 2014 1:33:41 PM Alan Smith wrote: It may be a tad early yet, but I am about to the point in my comfort with Rivendell to start thinking about backups. I know this may not be the most common way, but we have our reasons, and I'd like to duplicate this behavior if I can: Our current automation system [MS-DOS based] contains two non-raid, but mirrored hard drives. On a schedule that we specify, it copies the new files, deletes the obsolete ones, etc. Is there a way I can do this with Rivendell? Again, I realize most people are probably using a central server with a redundant server to house all station audio, clocks, etc, but I am comfortable with the way we are set up now. Besides we have quite a few stations spread out, so a central file store wouldn't work anyway in most of our cases. With my very limited knowledge of linux, I thought perhaps running dd with chron might be the ticket, but coming from a Windows guy, I don't know how linux behaves when attempting to copy files in use. Thanks for any suggestions. Oh, and all my Rivendell installs are/will be based on the Paravel Appliances. I have one V1 appliance in the field now, and currently working on my first V2. -Alan PS We actually had a hard drive die this past Saturday on one of our DOS stations. I just simply powered down, unlocked the bad drive from its removable cage, powered back up, and we were back in business. ___ Rivendell-dev mailing list Rivendell-dev@lists.rivendellaudio.org http://caspian.paravelsystems.com/mailman/listinfo/rivendell-dev ___ Rivendell-dev mailing list Rivendell-dev@lists.rivendellaudio.org http://caspian.paravelsystems.com/mailman/listinfo/rivendell-dev
[RDD] Redundant Hard Drive/Backup
It may be a tad early yet, but I am about to the point in my comfort with Rivendell to start thinking about backups. I know this may not be the most common way, but we have our reasons, and I'd like to duplicate this behavior if I can: Our current automation system [MS-DOS based] contains two non-raid, but mirrored hard drives. On a schedule that we specify, it copies the new files, deletes the obsolete ones, etc. Is there a way I can do this with Rivendell? Again, I realize most people are probably using a central server with a redundant server to house all station audio, clocks, etc, but I am comfortable with the way we are set up now. Besides we have quite a few stations spread out, so a central file store wouldn't work anyway in most of our cases. With my very limited knowledge of linux, I thought perhaps running dd with chron might be the ticket, but coming from a Windows guy, I don't know how linux behaves when attempting to copy files in use. Thanks for any suggestions. Oh, and all my Rivendell installs are/will be based on the Paravel Appliances. I have one V1 appliance in the field now, and currently working on my first V2. -Alan PS We actually had a hard drive die this past Saturday on one of our DOS stations. I just simply powered down, unlocked the bad drive from its removable cage, powered back up, and we were back in business. ___ Rivendell-dev mailing list Rivendell-dev@lists.rivendellaudio.org http://caspian.paravelsystems.com/mailman/listinfo/rivendell-dev
