Need help with backup plan
Right now I back up my Netware server with Arcserve to a single tape. I have 10 tapes. The system is set up to accept any tape and run a full backup. What we do is keep the tapes in rotation and if someone forgets to bring a tape back from off site it is no big deal we just use the next one. Arcserve assigns each tape a new ID when it uses it. We write the date of the backup on the case of each tape. If a restore is needed we figure out what date we want and insert the tape. Arcserve will tell us the tape ID and then we can restore from that session. Now I'm switching to FreeBSD and have a working Amanda install with 10 tapes. I have yet to configure the cycle, etc... as I'm not sure how to make it work like Arcserve did. Can Amanda be made to work like that?
Re: Need help with backup plan
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On Tuesday, 31.08.2004 at 10:42 -0400, Joe Konecny wrote: Right now I back up my Netware server with Arcserve to a single tape. I have 10 tapes. The system is set up to accept any tape and run a full backup. What we do is keep the tapes in rotation and if someone forgets to bring a tape back from off site it is no big deal we just use the next one. Arcserve assigns each tape a new ID when it uses it. We write the date of the backup on the case of each tape. If a restore is needed we figure out what date we want and insert the tape. Arcserve will tell us the tape ID and then we can restore from that session. Now I'm switching to FreeBSD and have a working Amanda install with 10 tapes. I have yet to configure the cycle, etc... as I'm not sure how to make it work like Arcserve did. Can Amanda be made to work like that? Won't comment specifically for your case but most questions of Can Amanda be made to work like that? are normally not the right thing to ask. Find out how AMANDA actually *does* it and then fit in with its way of doing things. AMANDA is good at scheduling backups. Dave. - -- Dave Ewart [EMAIL PROTECTED] Computing Manager, Epidemiology Unit, Oxford Cancer Research UK PGP: CC70 1883 BD92 E665 B840 118B 6E94 2CFD 694D E370 -BEGIN PGP SIGNATURE- Version: GnuPG v1.2.4 (GNU/Linux) iD8DBQFBNI/fbpQs/WlN43ARAlVzAKC70NcgufKVQtbeK4DLGacf0AMzYgCaA8Gi xZjFI6PMHi4RKbfyZMmf19g= =Qo1J -END PGP SIGNATURE-
Re: Need help with backup plan
On Tue, Aug 31, 2004 at 10:42:58AM -0400, Joe Konecny wrote: The system is set up to accept any tape and run a full backup. What we do is keep the tapes in rotation and if someone forgets to bring a tape back from off site it is no big deal we just use the next one. This is the interesting part. Amanda tries to protect you from accidentally overwriting backups that you wanted to keep. To that end, it won't accept any tape, but only tapes whose contents it considers to have expired. That's what the whole tapecycle thing's about. To get what you want, you'll have to defeat this protection. Typically, people have C tapes, and set tapecycle to C. Under those conditions (once you've reached equilibrium by running through all the tapes once) Amanda refuses to write to the C-1 most recently used tapes; it will only accept the least recently used tape, C, or a new tape that's never been written to. That's the situation I'm familiar with. But suppose you have more than tapecycle tapes, i.e. you have T tapes for some TC. I believe (please correct me if necessary, folks) that in this case, Amanda will still refuse to write to the C-1 most recently used tapes, but will accept any of the older ones, i.e. any of tapes C through T. It'll say The next tape Amanda expects to use is: [label for tape T], but that's not quite true; in fact, it'll accept any tape that isn't in the range [1, C-1]. So, to get the behaviour you have now, I think you can just set tapecycle to 1. But I don't know for sure, because I'd never do that :-) Personally, I *like* it that Amanda protects me from my own mistakes (forgetting to change the tape, or mounting the wrong one, for example). Of course, you could use a hybrid approach and get yourself some protection against accidents, but also some of the flexibility you want, simply by setting tapecycle to something like 5; then, the latest few tapes would be protected, but any of the older ones would do. Arcserve assigns each tape a new ID when it uses it. This, Amanda won't do. You assign each tape an ID (i.e. label) when you amlabel it, and Amanda uses that same label forevermore (unless you manually relabel the tape of course, but Amanda discourages that). That difference in detail shouldn't cause any problem in meeting your basic requirement, though. We write the date of the backup on the case of each tape. If a restore is needed we figure out what date we want and insert the tape. Arcserve will tell us the tape ID and then we can restore from that session. In the Amanda world, you wouldn't need to write the backup date on the case. To restore, you'd: - figure out what date you want - use amrecover's setdate command, specifying that date - amrecover would tell *you* which tape to mount, asking for it by its label - should you ever switch to doing incremental backups instead of full ones every night, or should your backups grow to more than one tape per run, amrecover might need more than one tape to do a multi-file restore; it'll ask for each one in turn, again by label -- | | /\ |-_|/ Eric Siegerman, Toronto, Ont.[EMAIL PROTECTED] | | / It must be said that they would have sounded better if the singer wouldn't throw his fellow band members to the ground and toss the drum kit around during songs. - Patrick Lenneau
Re: Need help with backup plan
Gavin Henry wrote: Amanda is much better than that. It will but the backup in a holding disk i.e. folder/partition until space runs out or hits the limit you set. It will not overwrite another tape unless you force it by hand, which is good. You don't have to figure out which tape, amanda tells you. You can even browse the backup files like a normal Unix system and restore per file. You have not spec'd how often you backup either, how much data and much more if we need to help you. Read the docs first and look at this site: http://www.oops.co.at/AMANDA-docs/index.html So... is amanda really not made for disaster recovery? That's great that it will back up to a holding disk with no tape but if the hard drive dies it's rather meaningless.
Re: Need help with backup plan
On Tue, Aug 31, 2004 at 03:49:49PM -0400, Joe Konecny wrote: Gavin Henry wrote: Amanda is much better than that. It will but the backup in a holding disk i.e. folder/partition until space runs out or hits the limit you set. It will not overwrite another tape unless you force it by hand, which is good. You don't have to figure out which tape, amanda tells you. You can even browse the backup files like a normal Unix system and restore per file. You have not spec'd how often you backup either, how much data and much more if we need to help you. Read the docs first and look at this site: http://www.oops.co.at/AMANDA-docs/index.html So... is amanda really not made for disaster recovery? That's great that it will back up to a holding disk with no tape but if the hard drive dies it's rather meaningless. I don't understand the leap you've made here. The holding disk is a buffer to allow multiple items to be backed up simultaneously but with a single stream to tape. If no usable tape is available amanda will still do the backups but leave them in the buffer (holding disk) for later taping. Usable here can mean many things including a tape not part of amanda's collection (do you really want it to trash someone else's valuable data?), an amanda tape not permitted to be overwritten (admin controllable), or even a broken drive or no tape inserted. How does that imply not for disaster recovery? What would your current system do if the tape jammed? Would it still backup? True, if you encountered a situation where you did not put in a tape AND the holding disk died you would not have a backup. Is that your concern? BTW some amanda installations use cheaper, separate drives for the holding disk. On my primary home office system, the 4 system drives are scsi. My holding disk is an IDE drive bigger than the sum of the scsi drives. Once I went on a week long trip forgetting to insert a new set of tapes. Came home, inserted the tapes and flushed a weeks worth of normal backups on to several tapes. -- Jon H. LaBadie [EMAIL PROTECTED] JG Computing 4455 Province Line Road(609) 252-0159 Princeton, NJ 08540-4322 (609) 683-7220 (fax)
Re: Need help with backup plan
Joe Konecny wrote: Gavin Henry wrote: Amanda is much better than that. It will but the backup in a holding disk i.e. folder/partition until space runs out or hits the limit you set. It will not overwrite another tape unless you force it by hand, which is good. You don't have to figure out which tape, amanda tells you. You can even browse the backup files like a normal Unix system and restore per file. You have not spec'd how often you backup either, how much data and much more if we need to help you. Read the docs first and look at this site: http://www.oops.co.at/AMANDA-docs/index.html So... is amanda really not made for disaster recovery? That's great that it will back up to a holding disk with no tape but if the hard drive dies it's rather meaningless. Umm.. You're supposed to flush the holdingdisk to tape as soon as you realise that you forgot to change tapes the other day. Then you are safe..
Re: Need help with backup plan
Jon LaBadie wrote: So... is amanda really not made for disaster recovery? That's great that it will back up to a holding disk with no tape but if the hard drive dies it's rather meaningless. I don't understand the leap you've made here. The holding disk is a buffer to allow multiple items to be backed up simultaneously but with a single stream to tape. If no usable tape is available amanda will still do the backups but leave them in the buffer (holding disk) for later taping. Usable here can mean many things including a tape not part of amanda's collection (do you really want it to trash someone else's valuable data?), an amanda tape not permitted to be overwritten (admin controllable), or even a broken drive or no tape inserted. How does that imply not for disaster recovery? What would your current system do if the tape jammed? Would it still backup? True, if you encountered a situation where you did not put in a tape AND the holding disk died you would not have a backup. Is that your concern? BTW some amanda installations use cheaper, separate drives for the holding disk. On my primary home office system, the 4 system drives are scsi. My holding disk is an IDE drive bigger than the sum of the scsi drives. Once I went on a week long trip forgetting to insert a new set of tapes. Came home, inserted the tapes and flushed a weeks worth of normal backups on to several tapes. It's because of what I am used to... Right now Arcserve backs up my Netware server as long as there is a tape in the drive. To me that's ideal since the data collection in our factory runs 24-7. Even on Sat and Sun when I'm not there the tape is still over written with the latest data. If Amanda does only copies the data to a holding area and we lose a drive on Sunday night I have massive problems. That's what I call a disaster and I don't see how amanda can help me there. I do want to make it work however.
Re: Need help with backup plan
On Tue, Aug 31, 2004 at 04:29:43PM -0400, Joe Konecny wrote: Jon LaBadie wrote: So... is amanda really not made for disaster recovery? That's great that it will back up to a holding disk with no tape but if the hard drive dies it's rather meaningless. How does that imply not for disaster recovery? What would your current system do if the tape jammed? Would it still backup? True, if you encountered a situation where you did not put in a tape AND the holding disk died you would not have a backup. Is that your concern? It's because of what I am used to... Right now Arcserve backs up my Netware server as long as there is a tape in the drive. To me that's ideal since the data collection in our factory runs 24-7. Even on Sat and Sun when I'm not there the tape is still over written with the latest data. If Amanda does only copies the data to a holding area and we lose a drive on Sunday night I have massive problems. That's what I call a disaster and I don't see how amanda can help me there. I do want to make it work however. For the disaster you feel is possible several things have to occur at the same time. 1. Your tape or tapedrive fails, or you do not put in a suitable tape - though as has been said by another, with a tapecycle of 1, you could override that protection. 2. A drive containing valuable data would have to fail 3. The drive containing your holding disk would have to fail. Usually this is not the systems primary data drive. And it can also be spread over multiple drives so if one fails others will do the job. What about your current system. Fine, if the wrong tape is inserted you still get a current backup. Same with amanda. It is just on the holding disk. And I'd be worried about something else in your current system. Presently I'm starting to wear out tapes. Suppose you forgot to change a tape, I'm guessing the tape left in there has the most current backup and is about to be overwritten by the new backup. What if the tape fails, or the backup fails in the middle for some other reason. You will have neither todays nor the last backup. -- Jon H. LaBadie [EMAIL PROTECTED] JG Computing 4455 Province Line Road(609) 252-0159 Princeton, NJ 08540-4322 (609) 683-7220 (fax)
Re: Need help with backup plan
On Tue, Aug 31, 2004 at 04:29:43PM -0400, Joe Konecny wrote: To me that's ideal since the data collection in our factory runs 24-7. Even on Sat and Sun when I'm not there the tape is still over written with the latest data. If Amanda does only copies the data to a holding area and we lose a drive on Sunday night I have massive problems. Consider your current Arcserve scheme. If you lose the data-collection drive during a weekend backup run, your recent data is toast. The only backup since Thursday night has been destroyed (partially overwritten) along with the live data. Low probability, but it could happen. Now consider what we're suggesting -- Amanda backing up to holding disk, and the backups being flushed to tape on Monday. To lose the data, you'd have to lose both the live-data disk *and* the holding disk, although the restriction that it has to happen during a backup run doesn't apply. Losing a disk at some point over the weekend is a higher probability than losing it during the backup window, but losing both of them at once is *much less* likely ... unless there's a fire in the machine room or similar catastrophe over the weekend, in which case your current tape backup, which was still in the drive, is toast too -- under either scheme -- and again your most recent backup is the one from Thursday night. That's what I call a disaster and I don't see how amanda can help me there. Amanda is approximately as able to help there as your current scheme (equal, or a little better, but unless I'm missing something, not a lot better -- the improvement is precisely the difference in probability of data loss between the two schemes.) Amanda can also guard against failure modes that your current scheme doesn't guard against -- or, as I've described, it can be configured not to guard against them, if that better suits your requirements. -- | | /\ |-_|/ Eric Siegerman, Toronto, Ont.[EMAIL PROTECTED] | | / It must be said that they would have sounded better if the singer wouldn't throw his fellow band members to the ground and toss the drum kit around during songs. - Patrick Lenneau
Re: Need help with backup plan
On Tue, Aug 31, 2004 at 10:42:27PM +0200, Andreas Sundstrom wrote: I think that if things are so critical I would seriously investigate in purchasing a tape changer. Indeed! Even better, though perhaps impractical, would be to back the data up across the network to a remote site. Rsync might be a useful tool for this (http://samba.anu.edu.au/rsync/). It's smart enough to only copy the changed portions of a file, so even if your data is a single huge database, rsync might be able to back it up in reasonable time every time but the first. -- | | /\ |-_|/ Eric Siegerman, Toronto, Ont.[EMAIL PROTECTED] | | / It must be said that they would have sounded better if the singer wouldn't throw his fellow band members to the ground and toss the drum kit around during songs. - Patrick Lenneau
Re: Need help with backup plan
There's a little bit of culture clash going on here, I think :-) On Tue, Aug 31, 2004 at 04:29:43PM -0400, Joe Konecny wrote: Even on Sat and Sun when I'm not there the tape is still over written with the latest data. The *last* thing I would *ever* want to do is to overwrite my most recent good backup. What if the current backup fails? [*] Then I've lost them both. The very idea makes me cringe! Given a choice between (a) overwriting last night's backup and (b) failing to take tonight's backup, I'd reluctantly choose the latter. Admittedly, your site's needs might be different from ours, such that a choice that none of us would make, is the right one in your case. But really, I'd much prefer not to have to make that choice at all -- neither (a) nor (b) is very appealing -- and Amanda, used as we've described, gets me out of having to take either risk. [*] System crash, bug in the backup system (Amanda or otherwise), the heads need cleaning, the tape goes bad, the data won't fit on the tape, an attempt to back up a file as it's being written leads to an inconsistent image on the tape, or any of a dozen other failure modes, many of which are orders of magnitude more likely than a disk crash. If Amanda does only copies the data to a holding area and we lose a drive on Sunday night I have massive problems. That's what I call a disaster It's only a disaster if the holding area is on the same physical drive as the live data. So don't do that! Besides the robustness problem you've mentioned, it's also bad for backup performance. Put it on its own spindle. -- | | /\ |-_|/ Eric Siegerman, Toronto, Ont.[EMAIL PROTECTED] | | / It must be said that they would have sounded better if the singer wouldn't throw his fellow band members to the ground and toss the drum kit around during songs. - Patrick Lenneau
Re: Need help with backup plan
Eric Siegerman wrote: There's a little bit of culture clash going on here, I think :-) On Tue, Aug 31, 2004 at 04:29:43PM -0400, Joe Konecny wrote: Even on Sat and Sun when I'm not there the tape is still over written with the latest data. The *last* thing I would *ever* want to do is to overwrite my most recent good backup. What if the current backup fails? [*] Then I've lost them both. The very idea makes me cringe! Given a choice between (a) overwriting last night's backup and (b) failing to take tonight's backup, I'd reluctantly choose the latter. Admittedly, your site's needs might be different from ours, such that a choice that none of us would make, is the right one in your case. But really, I'd much prefer not to have to make that choice at all -- neither (a) nor (b) is very appealing -- and Amanda, used as we've described, gets me out of having to take either risk. [*] System crash, bug in the backup system (Amanda or otherwise), the heads need cleaning, the tape goes bad, the data won't fit on the tape, an attempt to back up a file as it's being written leads to an inconsistent image on the tape, or any of a dozen other failure modes, many of which are orders of magnitude more likely than a disk crash. If Amanda does only copies the data to a holding area and we lose a drive on Sunday night I have massive problems. That's what I call a disaster It's only a disaster if the holding area is on the same physical drive as the live data. So don't do that! Besides the robustness problem you've mentioned, it's also bad for backup performance. Put it on its own spindle. I guess that is my problem. I am currently configured to have the holding disk on the same physical drive. The best option appears to be to purchase a new drive. One question though... I assume the new drive should be at lease as big as the one I'm backing up?
Re: Need help with backup plan
On Tue, Aug 31, 2004 at 09:20:29PM -0400, Joe Konecny wrote: I guess that is my problem. I am currently configured to have the holding disk on the same physical drive. The best option appears to be to purchase a new drive. One question though... I assume the new drive should be at lease as big as the one I'm backing up? If you are trying to eliminate the lost data scenario, at least as big as your biggest nightly backup. (times as many days of tapeless backups as you might want to prepare for) -- Jon H. LaBadie [EMAIL PROTECTED] JG Computing 4455 Province Line Road(609) 252-0159 Princeton, NJ 08540-4322 (609) 683-7220 (fax)
