Re: [SLUG] Re: Backup notes from Mary's talk (28 Nov)
On 29/11/2008, at 9:59 AM, Mary Gardiner wrote: On 2008-11-28, Michael Chesterton <[EMAIL PROTECTED]> wrote: Very minor nitpick, rsync can save older data, and put them in whatever dir you want. before rdiff-backup i used to use rsync --backup --backup-dir=$date- based-dir Thanks for that: I am guessing from the rsync man page though that it isn't particularly careful to only store the differences and therefore may take up more space? Right, if a file has changed, it copies the unmodified file to -- backup-dir then syncs the file. If a file hasn't changed, it doesn't get copied to --backup-dir. -- http://chesterton.id.au/blog/ http://barrang.com.au/ -- SLUG - Sydney Linux User's Group Mailing List - http://slug.org.au/ Subscription info and FAQs: http://slug.org.au/faq/mailinglists.html
[SLUG] Re: Backup notes from Mary's talk (28 Nov)
On 2008-11-28, Michael Chesterton <[EMAIL PROTECTED]> wrote: > Very minor nitpick, rsync can save older data, and put them in > whatever dir you want. > > before rdiff-backup i used to use rsync --backup --backup-dir=$date- > based-dir Thanks for that: I am guessing from the rsync man page though that it isn't particularly careful to only store the differences and therefore may take up more space? There are some alternative rsync-based backup solutions that try and make the *entire* series of backups available as a file structure, wasting as little space as possible by hard-linking: http://www.rsnapshot.org/ is the one I've heard of most, http://www.nongnu.org/rdiff-backup/related.html under "rsync-based scripts" has others. For me I've generally found that I pretty much always want the most recent or next-to-most-recent backup, so I haven't needed this feature. -Mary -- SLUG - Sydney Linux User's Group Mailing List - http://slug.org.au/ Subscription info and FAQs: http://slug.org.au/faq/mailinglists.html
Re: [SLUG] Recovering data from failed hard drive
On Friday 28 November 2008 21:58:01 [EMAIL PROTECTED] wrote: > My two-week-old sole hard drive has failed. > > I know this is more than a software thing, because the drive's not found > either during boot-up or by fdisk. The clincher is that it keeps making a > sound like the sound that you hear from someone's iPod earphones when > you're sitting next to them! > > I believe that there are places that recover data from dead hard drives. > > Could anyone tell me the name of any reputable places in Sydney that do > such a thing? Is the process expensive? Would it matter that the data was > created using Linux? > > I'd be grateful for any pointers. > > Thanks for reading this. See the topic of Mary's talk :-) One of the local engineering companies just had a sydney company do recovery on a 4T raid array. The bill was just over $1 million. That was on working drives. I'd welcome learning better, but methinks the munnie value is $1000s not $100s. James -- SLUG - Sydney Linux User's Group Mailing List - http://slug.org.au/ Subscription info and FAQs: http://slug.org.au/faq/mailinglists.html
Re: [SLUG] Backup notes from Mary's talk (28 Nov)
On 28/11/2008, at 11:56 PM, Mary Gardiner wrote: See http://jwz.livejournal.com/801607.html for someting similar (and the partial inspiration for the talk), although rsync doesn't save older data, which I definitely recommend. Very minor nitpick, rsync can save older data, and put them in whatever dir you want. before rdiff-backup i used to use rsync --backup --backup-dir=$date- based-dir -- http://chesterton.id.au/blog/ http://barrang.com.au/ -- SLUG - Sydney Linux User's Group Mailing List - http://slug.org.au/ Subscription info and FAQs: http://slug.org.au/faq/mailinglists.html
Re: [SLUG] Backup notes from Mary's talk (28 Nov)
Mary Gardiner wrote: 2. Install the program called rdiff-backup. All excellent suggestions Mary. Sorry I missed the talk. As an additions to rdiff-backup, people might benefit from using backupninja. Its a front end to rdiff-backup and other backup tools (like mysqldump). That makes it easier to configure. http://riseuplabs.org/backupninja/ Also make sure to check out the rdiff-backup wiki. there is a ton of scripts and hints and tips on there to make using rdiff-backup easier. The rdiff-backup list has been working hard to make rdiff-backup work on windows natively now too. We have used winsshd to pull backups off windows boxes with rdiff-backup successfully. http://www.bitvise.com/winsshd (not Free) CAUTION 2: The remote backup space is more than a little crowded right now. Keep an eye on your provider, some are undoubtedly headed for failure. These guys have been active on the rdiff-backup mailing lists in the past http://rsync.net/ and support rdiff-backup as a backup target. I don't actually use them, but they seem to have their stuff together when it comes to offsite backups. Again, I second all the recommendations in this talk - it matches my commercial and personal experience very closely :) thanks Dave -- SLUG - Sydney Linux User's Group Mailing List - http://slug.org.au/ Subscription info and FAQs: http://slug.org.au/faq/mailinglists.html
[SLUG] Backup notes from Mary's talk (28 Nov)
Hi all, As promised here are the backup notes from my talk at SLUG on 28 Nov. Materials related to this talk are at http://users.puzzling.org/users/mary/Presentations/SLUG2008/ (including a version of these notes). A note about style: this is a set of recommendations purely based on the fact that I have both backed my home data up AND recovered it. And having some working backup regime is better than none. I don't claim this is the One Best Way, merely One of the Adequate Ways That Isn't Entirely Maddening. A note about me: I am sadly short of time at the moment and will not participate in the thread following this post (should there be one), and I can't give one-on-one help to design your personal backup regime. Sorry about that: hopefully slug@slug.org.au can help you out. The talk was on backups for home users. It doesn't cover mission-critical or business-grade backups. --- The magical 10 second version --- If you don't have backups, you should. Here's how: 1. Go out, right now, and buy an external hard drive as big as, or bigger than your main hard drive. Yep, there is no free lunch. 2. Install the program called rdiff-backup. 3. Plug in the drive and run: sudo rdiff-backup --exclude-other-filesystems ::/ ::/media/disk (/media/disk being the place your external drive mounted, under Ubuntu, substitute as needed) 4. Run that as often as you can. (Every so often, run "sudo rdiff-backup --force --remove-older-than 60D ::/media/disk" or similar to delete very old backups.) See http://jwz.livejournal.com/801607.html for someting similar (and the partial inspiration for the talk), although rsync doesn't save older data, which I definitely recommend. --- More about rdiff-backup --- Do check out the webpage and "man rdiff-backup" for full details. http://www.nongnu.org/rdiff-backup/ In summary, it's 'reverse' increments, if you will. That is, you can get the most recent backup just by looking at the filesystem under /media/disk. Older versions are recovered by rdiff-backup applying older and older chnages incrementally to the files, and are recovered like so: sudo rdiff-backup -r 1D /media/disk/path-to-file [destination you'd like to restore to] --- Why you need backups --- You may not want to protect against all of these things: some of them are expensive or time-consuming to protect against. But consider these risks when deciding on your backup regime. 1. Accidental deletion: very common. An on-site backup is good enough to recover from this. 2. Media failure (dead hard drive). This will happen to you, sooner or later. You may or may not get any warning. An on-site backup *on a different disk* is good enough to recover from this. Not a different partition, a different *disk*. This is the only one RAID helps with too provided that (a) you have a full mirror on the other disk(s) in the array and (b) you don't stuff it up somehow and set the new empty disk as the master. RAID is not a substitute for backups. *Not* a substitute for backups. It won't help with 1, 3, 4 or 5. 3. Software failure. Say some bit of code, from the drive firmware to the filesystem to the end user software (eg GIMP) has a bug in it and writes out your data incorrectly. In most cases this is rather like accidental deletion, but if the bug is very low level (kernel) it may affect the backup too. At the very least, have your backup drive be not the same manufacturer and model as your main drive. This makes them less likely to share the same bugs and less likely to fail at the same time. 4. Provider failure. You have uploaded your valuble data to Flickr, LiveJournal, WordPress.com etc etc. They go bust, and their creditors swoop in, turn their machines off and sell them for scrap parts. This really happens, see http://blogs.zdnet.com/digitalcameras/?p=362 for an example. Smaller examples are the occasional data loss that a lot of web services, up to and including those run by Google, have. 5. Massive local failure. Flood, fire, surge: we had victims of two of these at the meeting. And someone who had had all their computer equipment stolen in one go. To recover from these you need an(other) backup, as far away from your main data store as you can. At least in a different suburb. Another country is entirely possible these days, if you have broadband. --- Media recommendations --- For home users, get another hard drive and backup to that. Optical media: no. CDs and DVDs are too small for most people now. You will have to insert at least 5 of the things for a full backup cycle. So that's boring and dull, so you'll never do it. Also, they have a short-ish lifespan and testing their backup goodness is even *more* boring and dull, so you definitely won't ever do that. Solid state media: no. Consumer grade SSDs are really really unreliable right now. You need to back them up, not use them *for* your backups! See h
Re: [SLUG] Recovering data from failed hard drive
On 28/11/2008, at 8:02 PM, Ben Donohue wrote: hard drives very commonly won't spin up. if you can get them to spin up then they tend to allow you to get all the data off. The way to get it to spin up is to remove it from the computer, attach the cables, turn on the computer and rotate the drive in your hand in the same direction (back and forth) as the platters spin. (don't bash the disk but be gentle) This frees up the disks and they start spinning. I've had very many times of spinning up disks like this that have failed and have got all the data off. I've heard with some types of failures, sealing the drive in air tight plastic and putting it in the freezer resurrects it for a while. Research before you do it, though. I'm not sure if it will help in this case. -- http://chesterton.id.au/blog/ http://barrang.com.au/ -- SLUG - Sydney Linux User's Group Mailing List - http://slug.org.au/ Subscription info and FAQs: http://slug.org.au/faq/mailinglists.html
Re: [SLUG] Recovering data from failed hard drive
Hi Leslie, hard drives very commonly won't spin up. if you can get them to spin up then they tend to allow you to get all the data off. The way to get it to spin up is to remove it from the computer, attach the cables, turn on the computer and rotate the drive in your hand in the same direction (back and forth) as the platters spin. (don't bash the disk but be gentle) This frees up the disks and they start spinning. I've had very many times of spinning up disks like this that have failed and have got all the data off. You can get accessory USB to SATA or IDE cables and mount the disk as a USB drive. These cost in the range of $10-$20 and worth a shot rather than $$$s for professional data recovery. Not sure about SCSI. So try getting the disk to spin first. Ben Stumblng Tumblr wrote: My two-week-old sole hard drive has failed. I know this is more than a software thing, because the drive's not found either during boot-up or by fdisk. The clincher is that it keeps making a sound like the sound that you hear from someone's iPod earphones when you're sitting next to them! I believe that there are places that recover data from dead hard drives. Could anyone tell me the name of any reputable places in Sydney that do such a thing? Is the process expensive? Would it matter that the data was created using Linux? I'd be grateful for any pointers. Thanks for reading this. Leslie -- SLUG - Sydney Linux User's Group Mailing List - http://slug.org.au/ Subscription info and FAQs: http://slug.org.au/faq/mailinglists.html
