Re: simple backup script
on Tue, Sep 02, 2003 at 01:15:10AM +0200, Marcus Schopen ([EMAIL PROTECTED]) wrote: > Hi, > > I'm looking for a simple backup script, which uses e.g. dd and > additionally does some error handling and mail notification. I use > amanda for my daily and weekly backups, but to feel more secure, I > installed a second harddrive in my server today. Now I'm looking for a > nice and secure script, which does a full backup of the first harddisk > each day. There's a short FAQ on GNU/Linux system backups you may find useful, at: http://kmself.home.netcom.com/Linux/FAQs/backups.html > Secondly I'd like to do a fullbackup of a remote server, but I'm not > sure what's the best way to do a e.g. a dd over the lokal network. May > be the obove script could use for both. Amanda, for starters. Peace. -- Karsten M. Self <[EMAIL PROTECTED]>http://kmself.home.netcom.com/ What Part of "Gestalt" don't you understand? We are the unwilling... led by the unqualified... to do the unnecessary... for the ungrateful... -- GI in Vietnam, 1970 pgp0.pgp Description: PGP signature
Re: simple backup script
Kirk Strauser wrote: At 2003-09-02T04:58:31Z, Jacob Anawalt <[EMAIL PROTECTED]> writes: I guess if your 2nd hard disk is the same size or larger than the first and you want an exact copy of everything then dd might be the way to do it. Umm, no. dd gives consistent backups *only if* the drive being copied not mounted writeable. If not, then there's a very significant possibility that the backup may not even be fsck'able. rsync is pretty handy for syncing live filesystems. Cool, thanks for the tip. I'll add that right next to "don't use dump" on my list of things to avoid. :) Jacob -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of "unsubscribe". Trouble? Contact [EMAIL PROTECTED]
Re: simple backup script
on Tue, Sep 02, 2003, Yves Goergen ([EMAIL PROTECTED]) wrote: > Why don't you just use RAID to mirror your harddisk? > > Saves you daily backups and gives you instant backup on failure. And IIRC > your system can keep on running 'on one tyre'. Mirrored RAID is not a backup solution. A backup is necessarily different to what you have; you've got what you've got, which is why you want something else off tape. Peter. -- [EMAIL PROTECTED] The IWETHEY project: http://www.iwethey.org pgp0.pgp Description: PGP signature
Re: simple backup script
At 2003-09-02T04:58:31Z, Jacob Anawalt <[EMAIL PROTECTED]> writes: > I guess if your 2nd hard disk is the same size or larger than the first > and you want an exact copy of everything then dd might be the way to do > it. Umm, no. dd gives consistent backups *only if* the drive being copied not mounted writeable. If not, then there's a very significant possibility that the backup may not even be fsck'able. rsync is pretty handy for syncing live filesystems. -- Kirk Strauser pgp0.pgp Description: PGP signature
Re: simple backup script - tar
On Tue, 2 Sep 2003, Marcus Schopen wrote: > > > Fault tollerance and data backups are not the same thing, they are just > > different components of a data protection policy. yup .. > > My advice would be (as a minumum) to take a full backup on Monday night and > > incrementals or differentials all other nights. Like I say, as a minumim. > > which programm do you use to do that? simplest way on sunday in cron: tar zcvf monday.7days.tgz ` find /home /etc -mtime -7 -type f ` on saturday night, do full backups in cron: tar zcvf /mnt/backup_disk `find /home /etc -type f -print ` - stick in symbolic links too - make a script .. even easier... does all the additional checking, and emailing, and failure notices and different backup media etc http://www.Linux-Backup.net - what you backup is strictly depending on you and your parition scheme and where you keep "user data" no point to backing up (system) stuff that is already backedup gazillion times on the internet c ya alvin -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of "unsubscribe". Trouble? Contact [EMAIL PROTECTED]
Re: simple backup script - options
On Tue, 2 Sep 2003, Marcus Schopen wrote: > > > > Why don't you just use RAID to mirror your harddisk? - dd is NOT backup ... its an easy way to get data corruptions when disk-A has badblocks that are different that the badblocks on target disk-B - luckily, badblocks are rare and few now days - and dd limits you to roughly the same size partions on both ends otherwise, you have unused/wasted space raid[n] is NOT backup - raid does protect you from downtime if a disk dies and you can keep going for a bit with the other disk - have another disk failure and lose everything on all disks unless you know how to recreate data from inodes raid1 does NOT save a copy of a file - erase it .. and its gone from both disks > The first harddisk is running in a raid (mirroring). But beside this > raid I'd like to have a backup to this second harddisk. Call me paranoid > if you like, but I feel better with this solution. for the paranoid .. - backup daily from the last full backup to "daily backupdisk" - backup 30 days every week to "backupdisk 30" - full backup every week to "full backup disk" - backup 90 days every month - to "backupdisk 90" dailybackup, backup-30, backup-90 and full backupdisks are all different disks on different server -- assume that last weeks and previous full backups was bad .. - how many files did you just lose various backup scripts http://www.Linux-Backup.net -- restore your machine from "bare metal" on an irrelgular basis to make sure "backups is working" diff "ls -laR /current_system" "ls -laR /restored_from_backup" ( or whatever way you wanna verify your restored system ) c ya alvin > > Saves you daily backups and gives you instant backup on failure. And IIRC > > your system can keep on running 'on one tyre'. > > Saluti, > Marcus > -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of "unsubscribe". Trouble? Contact [EMAIL PROTECTED]
Re: simple backup script
Jacob Anawalt wrote:
[...snip...]
if($base eq '/home' and $ar eq 'data')
{
#Special case for the /home/data archive, clean out old files
`find /home/data/mail/ -ctime +1 -exec rm {} \\;`;
`find /home/data/log/ -ctime +1 -exec rm {} \\;`;
`find /home/data/ -ctime +1 -path
"/home/data/source_*_*/delivery/*" -exec rm {} \\;`
}
Sorry, I thought I had pulled that part of the code out of my example.
If you do want to use this code, don't copy this part (especially if you
do have a user 'daa' with a directory source_*_*/delivery that you want
to keep stuff in)
That rdiff-backup program looks interesting. I'm checking it out.
Jacob
--
To UNSUBSCRIBE, email to [EMAIL PROTECTED]
with a subject of "unsubscribe". Trouble? Contact [EMAIL PROTECTED]
Re: simple backup script
I've read the other posts, but am starting back with your origional to keep your full questions. Marcus Schopen wrote: Hi, I'm looking for a simple backup script, which uses e.g. dd and additionally does some error handling and mail notification. I use amanda for my daily and weekly backups, but to feel more secure, I installed a second harddrive in my server today. Now I'm looking for a nice and secure script, which does a full backup of the first harddisk each day. I guess if your 2nd hard disk is the same size or larger than the first and you want an exact copy of everything then dd might be the way to do it. Then if hda dies you could plug-in hdb in its place and be going as of yesterday. I don't know what your goals are here. If it was just to save yourself from a disk crash, then the RAID options suggested seemed like a good idea. If it is to provide yet another backup because you don't trust your amanda backups or just because you want to script something, then I'd suggest a script using tar. You didn't mention dump, and that's probably a good thing: http://lwn.net/2001/0503/a/lt-dump.php3 Secondly I'd like to do a fullbackup of a remote server, but I'm not sure what's the best way to do a e.g. a dd over the lokal network. May be the obove script could use for both. Thanks, Marcus Ok, are you backing up the remote server to this new hard disk or are you backing up the server with the new hard disk to a remote location? I'm going to run with this idea: You've got a new hard disk that you'd like to use for a second backup system in your network, duplicating the effort of Amanda. (Although, if Amanda has the storage capacity to backup everything, couldn't you just backup it's backups?) You would like to see a script thaht does this I write my own backup scripts using Perl to call GNU tar. There's a lot you can do with GNU tar, and even their documentation recomends using a script to do your backup and restore operations rather than calling it by hand. You can get GNU tar to do full (of course), incremental, and differential backups (well others too, but they seem to fit within the previous three to me.) Lets say you're just backing up /home (expand this to whatever you want, but if you realy just want a full copy of your hard disk, maybe just use dd) and you want to keep all your changes so you can recover to it two weeks or two months ago (assuming you have been running the backup process for that long). First you need to make a 'full dump' backup of home. Run tar with the listed incremental option but the incremental file you specify shouldn't exist. This snapshot file will provide the baseline for future backups. Next you decide if you are going to be creating incremental or differential backups off of this base backup. Incremental means less archive space used, but that to restore from scratch you need every incremental stop point. For example if you started in January and are dong monthly incrementals then you would need to extract January (the full) Feb, Mar, Apr, May, Jun, Jul, and August to get to September then the latest daily incremental off of the August dump. Doing differential backups means more storage space but that you need only three extractions (under the method I will propose) to fully recover. Here is the backup 'game plan' I am proposing for archiving disk A to disk B which is much much larger than the data you are archiving from disk A. Disclaimers: My 'game plan' does not address off site storage which is very important. It is as 'try under your own risk' as I can make it, no warrenty. It doesn't ask you questions like "What are you backing up to protect yourself against anyway". It doesn't address file security and protection. It's just an answer to your question, except that it might not be as simple as you were asking. I am not addressing the remote backup, but there ought to be a way to do this with scp/ssh and add it to the script, or to a seperate script. I'm interested in hearing how others set that up because I may do it some day. Before you look at my script below, I encourage you to check out Adump or flexdump: Adump - http://www.davidb.org/adump/ <http://www.davidb.org/adump/> Flexbackup - http://flexbackup.sourceforge.net/ <http://flexbackup.sourceforge.net/> I do a full dump at least once a year. You can do it more often if you like/have space, but you'll have to move the old full dump archive's snapshot file out of the way so my script will think one hasn't been done. I do monthly differential backups off of the "yearly" full backup. I do daily differential backups off of the latest monthly backup and archive friday's daily differentials as the 'weekly' backup, keep
Re: simple backup script
On Tue, Sep 02, 2003 at 03:43:22AM +0200, Marcus Schopen wrote: > Joyce, Matthew wrote: > >Fault tollerance and data backups are not the same thing, they are just > >different components of a data protection policy. > > > >My advice would be (as a minumum) to take a full backup on Monday night and > >incrementals or differentials all other nights. Like I say, as a minumim. > > which programm do you use to do that? > rdiff-backup. It mirror directories by only coping changes and also creates reverse-diffs so you can retrieve any previous versions too. -- .''`. Jason Chambers <[EMAIL PROTECTED]> : :' : Registered linux user #271693 `. `'` `-http://www.debian.org/ - The Universal Operating System pgp0.pgp Description: PGP signature
RE: simple backup script
> -Original Message- > From: Marcus Schopen [mailto:[EMAIL PROTECTED] > Sent: Tuesday, 2 September 2003 11:59 AM > To: [EMAIL PROTECTED] > Subject: Re: simple backup script > > > Joyce, Matthew wrote: > [...] > > RAID only provides resilience against hardware failures, it > does not > > protect anyone from user errors or mishaps which occurred > yesterday, > > or last week, or last month. > > right!!! > > > Fault tollerance and data backups are not the same thing, they are > > just different components of a data protection policy. > > > > My advice would be (as a minumum) to take a full backup on Monday > > night and incrementals or differentials all other nights. > Like I say, > > as a minumim. > > which programm do you use to do that? > I would imagine TAR would be your friend. I do not have this knowledge to give you. However, I would consider separating the data a from the system and approach the backup of these data type differently. Perhaps a full system weekly (Sunday) and before any major system modification would be enough and incremental or differential for your data. um... a quick google http://www.biochemistry.unimelb.edu.au/pscotney/backup/Backup-HOWTO.html http://www.linux-backup.net and our very own Karsten M Self :) http://kmself.home.netcom.com/Linux/FAQs/backups.html -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of "unsubscribe". Trouble? Contact [EMAIL PROTECTED]
Re: simple backup script
On Tue, Sep 02, 2003 at 11:24:20AM +1000, Joyce, Matthew wrote: > RAID only provides resilience against hardware failures, it does not protect > anyone from user errors or mishaps which occurred yesterday, or last week, > or last month. It also doesn't protect you against the computer being struck by lightning, or the equivalent. Removable storage still has a very large place in the world, and it's not going away any time soon. -- Marc Wilson | War doesn't prove who's right, just who's left. [EMAIL PROTECTED] | -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of "unsubscribe". Trouble? Contact [EMAIL PROTECTED]
Re: simple backup script
Joyce, Matthew wrote: [...] RAID only provides resilience against hardware failures, it does not protect anyone from user errors or mishaps which occurred yesterday, or last week, or last month. right!!! Fault tollerance and data backups are not the same thing, they are just different components of a data protection policy. My advice would be (as a minumum) to take a full backup on Monday night and incrementals or differentials all other nights. Like I say, as a minumim. which programm do you use to do that? Ciao, Marcus -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of "unsubscribe". Trouble? Contact [EMAIL PROTECTED]
RE: simple backup script
> -Original Message- > From: Yves Goergen [mailto:[EMAIL PROTECTED] > Sent: Tuesday, 2 September 2003 9:43 AM > To: [EMAIL PROTECTED] > Subject: Re: simple backup script > > > On Tuesday, September 02, 2003 1:15 AM CET, Marcus Schopen wrote: > > Hi, > > > > I'm looking for a simple backup script, which uses e.g. dd and > > additionally does some error handling and mail notification. I use > > amanda for my daily and weekly backups, but to feel more secure, I > > installed a second harddrive in my server today. Now I'm > looking for a > > nice and secure script, which does a full backup of the > first harddisk > > each day. > > Why don't you just use RAID to mirror your harddisk? > > Saves you daily backups and gives you instant backup on > failure. And IIRC your system can keep on running 'on one tyre'. > RAID only provides resilience against hardware failures, it does not protect anyone from user errors or mishaps which occurred yesterday, or last week, or last month. Fault tollerance and data backups are not the same thing, they are just different components of a data protection policy. My advice would be (as a minumum) to take a full backup on Monday night and incrementals or differentials all other nights. Like I say, as a minumim. Matt -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of "unsubscribe". Trouble? Contact [EMAIL PROTECTED]
Re: simple backup script
On Tue, 2 Sep 2003 01:42:25 +0200, "Yves Goergen" <[EMAIL PROTECTED]> wrote in message <[EMAIL PROTECTED]>: > On Tuesday, September 02, 2003 1:15 AM CET, Marcus Schopen wrote: > > Hi, > > > > I'm looking for a simple backup script, which uses e.g. dd and > > additionally does some error handling and mail notification. I use > > amanda for my daily and weekly backups, but to feel more secure, I > > installed a second harddrive in my server today. Now I'm looking for > > a nice and secure script, which does a full backup of the first > > harddisk each day. ..'tar cf mail-spool-`date` /var/spool/mail' on Fridays and tar af etc on weekdays? Leaving out compression facilitates diff'ing etc, compression saves space and bandwidth but needs cpu oomph, YMWV. ..on completion, scp to an off-site backup box, businesses survive fires etc with off-site backup, and dies without. Script game. > Why don't you just use RAID to mirror your harddisk? > > Saves you daily backups and gives you instant backup on failure. And > IIRC your system can keep on running 'on one tyre'. ...such as /var/spool on ext3 on raid-1 going ro for a coupl'a days on a wee journal cock-up? "The best of 2 worlds!". ;-) ..monitoring and fix ideas? I mean, a kernel panic is a bit overkill when "the flat tire can be patched-n-pumped". ;-) -- ..med vennlig hilsen = with Kind Regards from Arnt... ;-) ...with a number of polar bear hunters in his ancestry... Scenarios always come in sets of three: best case, worst case, and just in case. -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of "unsubscribe". Trouble? Contact [EMAIL PROTECTED]
Re: simple backup script
Yves Goergen wrote: On Tuesday, September 02, 2003 1:15 AM CET, Marcus Schopen wrote: Hi, I'm looking for a simple backup script, which uses e.g. dd and additionally does some error handling and mail notification. I use amanda for my daily and weekly backups, but to feel more secure, I installed a second harddrive in my server today. Now I'm looking for a nice and secure script, which does a full backup of the first harddisk each day. Why don't you just use RAID to mirror your harddisk? The first harddisk is running in a raid (mirroring). But beside this raid I'd like to have a backup to this second harddisk. Call me paranoid if you like, but I feel better with this solution. Saves you daily backups and gives you instant backup on failure. And IIRC your system can keep on running 'on one tyre'. Saluti, Marcus -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of "unsubscribe". Trouble? Contact [EMAIL PROTECTED]
Re: simple backup script
On Tuesday, September 02, 2003 1:15 AM CET, Marcus Schopen wrote: > Hi, > > I'm looking for a simple backup script, which uses e.g. dd and > additionally does some error handling and mail notification. I use > amanda for my daily and weekly backups, but to feel more secure, I > installed a second harddrive in my server today. Now I'm looking for a > nice and secure script, which does a full backup of the first harddisk > each day. Why don't you just use RAID to mirror your harddisk? Saves you daily backups and gives you instant backup on failure. And IIRC your system can keep on running 'on one tyre'. -- Yves Goergen [EMAIL PROTECTED] Please don't CC me (causes double mails) -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of "unsubscribe". Trouble? Contact [EMAIL PROTECTED]
simple backup script
Hi, I'm looking for a simple backup script, which uses e.g. dd and additionally does some error handling and mail notification. I use amanda for my daily and weekly backups, but to feel more secure, I installed a second harddrive in my server today. Now I'm looking for a nice and secure script, which does a full backup of the first harddisk each day. Secondly I'd like to do a fullbackup of a remote server, but I'm not sure what's the best way to do a e.g. a dd over the lokal network. May be the obove script could use for both. Thanks, Marcus -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of "unsubscribe". Trouble? Contact [EMAIL PROTECTED]
