On 08/06/2012 10:42 PM, Nick Holland wrote: > On 08/06/12 17:22, Geoff Steckel wrote: >> Does anyone know what the current state of softraid 5 is? >> The man page says rebuild and scrub are not supported. >> The last checkin was about 6 months ago. > sounds like your question is answered. > Scrub and rebuild are critical for RAID5, if that wasn't obvious... > Play, write code, don't put into production yet. > >> Any information would be appreciated. >> I've got 3 or 4 terabytes that need a reliable home. >> And yes, RAID is no substitute for backups. >> One place I worked put 4 drives in a case with >> fans for 1. RAID go bye-bye. > Sometimes, the lack of ability to use your first choice of a design > causes you to look closer, think harder, and often you come up with a > better solution. > > For the amount of data you are talking, and the lack of other key words > like access time and such, I'm guessing you are looking at music, video > and picture-type files. Mostly static stuff. > > If your issue is "not losing data", and your data is mostly static, get > a few 2-3TB disks, break them up into 1TB partitions. Fill a chunk, > SHA256 all the files on that chunk, mark it "read only". Fill next > chunk, SHA256 all the files, mark it read only, etc. As the chunks are > filling, rsync them to another disk, preferably in another machine. > > Your actively filling chunk, maybe you want to make that RAID1 until it > is full, then copy it off to two separate chunks, and start over. > > Periodically, re-run your SHA256's against your RO files, looking for > "changed" data...and fix (from the other copy) if found. > > Note: this can give you an actual backup of your data. Good as a one > month rotation with monthly pulls? Of course not, but beats the heck > out of RAID(anything). > > Why "chunks" (partitions) of 1TB rather than one Huge Disk? Several > reasons: > * Encourages you to "lock" file systems and mount them only as read-only. > * Encourages you to PLAN for filled file systems. This file system WILL > fill in the near future. You will have to do something different in the > near future. Plan for it now. > * Makes upgrading storage easier: > * Install new disk. > * Point new files to go to new disk. > * if new disk is significantly bigger than old disk: > * at leisure, copy chunks from old disk to new disk. > * Verify successful copy > * remove old disk. > (note: 1TB takes a while to move. I don't care how you do it) > * Beats the heck out of copying all data from old to new system > and being down until it is done!! > * RO partitions contain and minimize some kinds of disasters. > > I did this some years back on an e-mail archive (actually, I used a > number of small arrays, rather than individual disks). I must say, > there was no question in my mind after running it through a number of > technology improvements and other "events", several small partitions > beat the heck out of one big array. Blew out a big chunk of the storage > at one point...no big deal, was restoring from (a snoot-load of) DVDs > while it was gathering more data at the same time -- downtime measured > in a small number of hours (and no lost data). > > In my day job, I do have the opportunity to use ZFS and other volume > managers and fancy file systems. For the most part, they just cover for > bad (or no) system design rather than solving problems that can't be > solved better in other ways. Not that I haven't had them help me out > (maybe even haul my ass out of the fire), but usually the message should > be, "your design sucked, you didn't know what you were doing, maybe you > should start over". > > Nick. Yes, this is all very sensible and I've done things that way before. I've already split out the (almost) static data onto multiple drives which has saved me several times. Even so, that data sometimes gets modified so it has to be carefully watched.
Unfortunately, the bulk of the data are not entirely or predictably RO, the rate of filling is unpredictable, and segmenting it into "chunks" would make it largely unusable. I'd have to rebalance fairly frequently and accidentally running out of space on a chunk would be very painful. With my current 2TB file system, I have to plan fairly carefully to avoid saving too many intermediate steps. The main failure mode I've seen is drive (not file system) failure so splitting a drive into multiple file systems hasn't been useful. There's a very large premium for me to use something which doesn't require installing and removing internal drives. Good SATA/SAS drive bays are very expensive. I've used external USB drives which are messy, slow, and less reliable than I'd like, but I already use a number of them and cycle through them as backups. A RAID 1 volume for online use looks like the best available solution to prevent loss of current data (since last backup). Thanks very much for your thoughts! Geoff
