Re: breaking up harddrives for gtar
On Tue, Aug 31, 2004 at 03:37:14PM -0700, Mike Fedyk wrote:
> Jon LaBadie wrote:
> >On Tue, Aug 31, 2004 at 04:57:42PM -0400, Jason Castonguay wrote:
> >
> >>This has been working pretty well, but I read somewhere that one can get
> >>even better results by breaking things up to as many small directories as
> >>one can. This makes sense to me, as it will allow amanda to schedule
> >
> >I do it with a set of disk list entries all with the same
> >starting directory but different uniq "tag" names. In most
> >I use the include directive (or include append) to list the
> >subdirectory or directories I want backed up by that DLE.
> >In one final DLE starting from the same directory I use the
> >exclude append directive to eliminate all the subdirs backed
> >up by the earlier mentioned DLEs. This serves as a catchall
> >for any dirs not specifically backed up by its own DLE.
> >
> Can you give an example of this?
>
> You have multiple entries in your disklist something like so?
>
> hostfoo /parent comp-tar-master
> hostfoo /parent/a comp-tar-child
> hostfoo /parent/b comp-tar-child
>
> And comp-tar-master excludes /parent/a and /parent/b?
>
> Otherwise, doesn't Amanda keep you from doing this?
>
> hostfoo /parent comp-tar-master
> hostfoo /parent comp-tar-child-a
> hostfoo /parent comp-tar-child-b
>
> which is the same as the previous except comp-tar-child-a includes
> /parent/a and comp-tar-child-b that includes /parent/b
>
> Doesn't Amanda prevent you from several DLEs with the same path
> ("/parent" in this case)?
>From the amanda(8) man page:
The disklist file determines which disks will be backed up
by Amanda. The file usually contains one line per disk:
hostname diskname [ diskdevice ] dumptype [ spindle [
interface ] ]
All pair [ hostname diskname ] must be unique.
By not reading the man page before posting, I refered to the
diskname as a "tag". If both diskname and diskdevice are
present, the second is the more traditional "starting dir"
and can appear multiple times. If only one is present,
it is both the diskdevice and diskname.
Thus, a revision of your sample disklist
hostfoo ParentCatchall /parent comp-tar-master
hostfoo ParentBigDir1 /parent comp-tar-child-a
hostfoo ParentBigDir2 /parent comp-tar-child-b
would be valid and each dumptype could have different specs
which would allow different includes/excludes.
The diskname is used in amanda reports. I'm not certain,
but it might be possible to use slashes in the name and
thus the above list might be valid this way too, but I'm
not 100% certain.
hostfoo /parent /parent comp-tar-master
hostfoo /parent/BigDir1 /parent comp-tar-child-a
hostfoo /parent/BigDir2 /parent comp-tar-child-b
However, I make use of an alternate form of the DLE whereby
you can customize each DLE individually.
hostfoo ParentBigDir1 /parent {
comp-tar
include ./BigDir1
}
hostfoo ParentBigDir2 /parent {
comp-tar
include ./BigDir2
}
hostfoo ParentCatchall /parent {
comp-tar
exclude./BigDir1
exclude append ./BigDir2
}
I changed the order as it make more sense to me to have the
catchall last.
HTH
--
Jon H. LaBadie [EMAIL PROTECTED]
JG Computing
4455 Province Line Road(609) 252-0159
Princeton, NJ 08540-4322 (609) 683-7220 (fax)
Re: Need help with backup plan
On Tue, Aug 31, 2004 at 09:20:29PM -0400, Joe Konecny wrote: > > I guess that is my problem. I am currently configured to have > the holding disk on the same physical drive. The best option > appears to be to purchase a new drive. One question though... > I assume the new drive should be at lease as big as the one > I'm backing up? > If you are trying to eliminate the lost data scenario, at least as big as your biggest nightly backup. (times as many days of tapeless backups as you might want to prepare for) -- Jon H. LaBadie [EMAIL PROTECTED] JG Computing 4455 Province Line Road(609) 252-0159 Princeton, NJ 08540-4322 (609) 683-7220 (fax)
Re: Need help with backup plan
Eric Siegerman wrote: There's a little bit of culture clash going on here, I think :-) On Tue, Aug 31, 2004 at 04:29:43PM -0400, Joe Konecny wrote: Even on Sat and Sun when I'm not there the tape is still over written with the latest data. The *last* thing I would *ever* want to do is to overwrite my most recent good backup. What if the current backup fails? [*] Then I've lost them both. The very idea makes me cringe! Given a choice between (a) overwriting last night's backup and (b) failing to take tonight's backup, I'd reluctantly choose the latter. Admittedly, your site's needs might be different from ours, such that a choice that none of us would make, is the right one in your case. But really, I'd much prefer not to have to make that choice at all -- neither (a) nor (b) is very appealing -- and Amanda, used as we've described, gets me out of having to take either risk. [*] System crash, bug in the backup system (Amanda or otherwise), the heads need cleaning, the tape goes bad, the data won't fit on the tape, an attempt to back up a file as it's being written leads to an inconsistent image on the tape, or any of a dozen other failure modes, many of which are orders of magnitude more likely than a disk crash. If Amanda does only copies the data to a holding area and we lose a drive on Sunday night I have massive problems. That's what I call a disaster It's only a disaster if the holding "area" is on the same physical drive as the live data. So don't do that! Besides the robustness problem you've mentioned, it's also bad for backup performance. Put it on its own spindle. I guess that is my problem. I am currently configured to have the holding disk on the same physical drive. The best option appears to be to purchase a new drive. One question though... I assume the new drive should be at lease as big as the one I'm backing up?
Re: Dump ordering optimisation documentation
Hi, Jon, on Dienstag, 31. August 2004 at 23:30 you wrote to amanda-users: JL> On Tue, Aug 31, 2004 at 02:14:40PM -0700, Mike Fedyk wrote: >> Is there any chance we can get this in the documentation? >> >> http://marc.theaimsgroup.com/?l=amanda-users&m=106277015010167&w=2 >> JL> I don't know of any email messages included with the amanda docs. This does not mean that there won't be messages included in the future. But the docs should definitely more than just a bunch of pasted emails. JL> Perhaps you could use that as the basis for a HOW-TO using the JL> taperalgo and dumporder params. Yes. I would suggest a new and shiny amanda.conf-HOWTO. This could be a part of it. Just take the current example/amanda.conf and make a HOWTO of it. I will be glad to help it getting done. This could also go into some kind of Tuning-HOWTO ... As always, suggestions are welcome -- best regards, Stefan Stefan G. Weichinger mailto:[EMAIL PROTECTED]
Re: Need help with backup plan
There's a little bit of culture clash going on here, I think :-) On Tue, Aug 31, 2004 at 04:29:43PM -0400, Joe Konecny wrote: > Even on Sat and Sun when I'm not > there the tape is still over written with the latest data. The *last* thing I would *ever* want to do is to overwrite my most recent good backup. What if the current backup fails? [*] Then I've lost them both. The very idea makes me cringe! Given a choice between (a) overwriting last night's backup and (b) failing to take tonight's backup, I'd reluctantly choose the latter. Admittedly, your site's needs might be different from ours, such that a choice that none of us would make, is the right one in your case. But really, I'd much prefer not to have to make that choice at all -- neither (a) nor (b) is very appealing -- and Amanda, used as we've described, gets me out of having to take either risk. [*] System crash, bug in the backup system (Amanda or otherwise), the heads need cleaning, the tape goes bad, the data won't fit on the tape, an attempt to back up a file as it's being written leads to an inconsistent image on the tape, or any of a dozen other failure modes, many of which are orders of magnitude more likely than a disk crash. > If Amanda does only copies the data to a holding area and > we lose a drive on Sunday night I have massive problems. > That's what I call a disaster It's only a disaster if the holding "area" is on the same physical drive as the live data. So don't do that! Besides the robustness problem you've mentioned, it's also bad for backup performance. Put it on its own spindle. -- | | /\ |-_|/ > Eric Siegerman, Toronto, Ont.[EMAIL PROTECTED] | | / It must be said that they would have sounded better if the singer wouldn't throw his fellow band members to the ground and toss the drum kit around during songs. - Patrick Lenneau
Re: breaking up harddrives for gtar
Jon LaBadie wrote:
On Tue, Aug 31, 2004 at 04:57:42PM -0400, Jason Castonguay wrote:
-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1
Hi,
Right now I am using amanda for a few different linux servers. My disklists
use the directories where different partitions are mounted. This has been
working pretty well, but I read somewhere that one can get even better
results by breaking things up to as many small directories as one can. This
makes sense to me, as it will allow amanda to schedule different level
backups the best she can even if it makes things perhaps more difficult to
keep track of.
So I collected on each client the names of the harddrives for it.
find / -type d -fprintf ~/amandalist "`hostname -f`\t\t%p\tcomp-user-tar\n"
and I have an idea to exclude all subdirectories in the dumptype, but then I
run into the problem of users creating and removing their own subdirectories
and then dealing with those. Do people who advocate this sort of setup
generate their disklist each time they run amanda? or do they not have to
deal with this problem? What is the best use of gtar?
I do it with a set of disk list entries all with the same
starting directory but different uniq "tag" names. In most
I use the include directive (or include append) to list the
subdirectory or directories I want backed up by that DLE.
In one final DLE starting from the same directory I use the
exclude append directive to eliminate all the subdirs backed
up by the earlier mentioned DLEs. This serves as a catchall
for any dirs not specifically backed up by its own DLE.
Can you give an example of this?
You have multiple entries in your disklist something like so?
hostfoo /parent comp-tar-master
hostfoo /parent/a comp-tar-child
hostfoo /parent/b comp-tar-child
And comp-tar-master excludes /parent/a and /parent/b?
Otherwise, doesn't Amanda keep you from doing this?
hostfoo /parent comp-tar-master
hostfoo /parent comp-tar-child-a
hostfoo /parent comp-tar-child-b
which is the same as the previous except comp-tar-child-a includes
/parent/a and comp-tar-child-b that includes /parent/b
Doesn't Amanda prevent you from several DLEs with the same path
("/parent" in this case)?
Mike
Re: Need help with backup plan
On Tue, Aug 31, 2004 at 10:42:27PM +0200, Andreas Sundstrom wrote: > I think that if things are so critical I would seriously > investigate in purchasing a tape changer. Indeed! Even better, though perhaps impractical, would be to back the data up across the network to a remote site. Rsync might be a useful tool for this (http://samba.anu.edu.au/rsync/). It's smart enough to only copy the changed portions of a file, so even if your data is a single huge database, rsync might be able to back it up in reasonable time every time but the first. -- | | /\ |-_|/ > Eric Siegerman, Toronto, Ont.[EMAIL PROTECTED] | | / It must be said that they would have sounded better if the singer wouldn't throw his fellow band members to the ground and toss the drum kit around during songs. - Patrick Lenneau
Re: Need help with backup plan
On Tue, Aug 31, 2004 at 04:29:43PM -0400, Joe Konecny wrote: > To me that's ideal since the data collection > in our factory runs 24-7. Even on Sat and Sun when I'm not > there the tape is still over written with the latest data. > If Amanda does only copies the data to a holding area and > we lose a drive on Sunday night I have massive problems. Consider your current Arcserve scheme. If you lose the data-collection drive during a weekend backup run, your recent data is toast. The only backup since Thursday night has been destroyed (partially overwritten) along with the live data. Low probability, but it could happen. Now consider what we're suggesting -- Amanda backing up to holding disk, and the backups being flushed to tape on Monday. To lose the data, you'd have to lose both the live-data disk *and* the holding disk, although the restriction that it has to happen during a backup run doesn't apply. Losing a disk at some point over the weekend is a higher probability than losing it during the backup window, but losing both of them at once is *much less* likely ... unless there's a fire in the machine room or similar catastrophe over the weekend, in which case your current tape backup, which was still in the drive, is toast too -- under either scheme -- and again your most recent backup is the one from Thursday night. > That's what I call a disaster and I don't see how amanda > can help me there. Amanda is approximately as able to help there as your current scheme (equal, or a little better, but unless I'm missing something, not a lot better -- the improvement is precisely the difference in probability of data loss between the two schemes.) Amanda can also guard against failure modes that your current scheme doesn't guard against -- or, as I've described, it can be configured not to guard against them, if that better suits your requirements. -- | | /\ |-_|/ > Eric Siegerman, Toronto, Ont.[EMAIL PROTECTED] | | / It must be said that they would have sounded better if the singer wouldn't throw his fellow band members to the ground and toss the drum kit around during songs. - Patrick Lenneau
Re: breaking up harddrives for gtar
On Tue, Aug 31, 2004 at 04:57:42PM -0400, Jason Castonguay wrote: > -BEGIN PGP SIGNED MESSAGE- > Hash: SHA1 > > Hi, > > Right now I am using amanda for a few different linux servers. My disklists > use the directories where different partitions are mounted. This has been > working pretty well, but I read somewhere that one can get even better > results by breaking things up to as many small directories as one can. This > makes sense to me, as it will allow amanda to schedule different level > backups the best she can even if it makes things perhaps more difficult to > keep track of. > > So I collected on each client the names of the harddrives for it. > > find / -type d -fprintf ~/amandalist "`hostname -f`\t\t%p\tcomp-user-tar\n" > > and I have an idea to exclude all subdirectories in the dumptype, but then I > run into the problem of users creating and removing their own subdirectories > and then dealing with those. Do people who advocate this sort of setup > generate their disklist each time they run amanda? or do they not have to > deal with this problem? What is the best use of gtar? > I do it with a set of disk list entries all with the same starting directory but different uniq "tag" names. In most I use the include directive (or include append) to list the subdirectory or directories I want backed up by that DLE. In one final DLE starting from the same directory I use the exclude append directive to eliminate all the subdirs backed up by the earlier mentioned DLEs. This serves as a catchall for any dirs not specifically backed up by its own DLE. -- Jon H. LaBadie [EMAIL PROTECTED] JG Computing 4455 Province Line Road(609) 252-0159 Princeton, NJ 08540-4322 (609) 683-7220 (fax)
Re: Dump ordering optimisation documentation
Jon LaBadie wrote: On Tue, Aug 31, 2004 at 02:14:40PM -0700, Mike Fedyk wrote: Is there any chance we can get this in the documentation? http://marc.theaimsgroup.com/?l=amanda-users&m=106277015010167&w=2 I don't know of any email messages included with the amanda docs. Perhaps you could use that as the basis for a HOW-TO using the taperalgo and dumporder params. Oh sorry, it looks like I wasn't specific enough. I meant to ask if something like this could be added to the current documentation that has been posted on the list lately, and maybe a parring down for the man page. Not including this email itself within the sources. Mike
Re: Dump ordering optimisation documentation
On Tue, Aug 31, 2004 at 02:14:40PM -0700, Mike Fedyk wrote: > Is there any chance we can get this in the documentation? > > http://marc.theaimsgroup.com/?l=amanda-users&m=106277015010167&w=2 > I don't know of any email messages included with the amanda docs. Perhaps you could use that as the basis for a HOW-TO using the taperalgo and dumporder params. -- Jon H. LaBadie [EMAIL PROTECTED] JG Computing 4455 Province Line Road(609) 252-0159 Princeton, NJ 08540-4322 (609) 683-7220 (fax)
Re: Need help with backup plan
On Tue, Aug 31, 2004 at 04:29:43PM -0400, Joe Konecny wrote: > Jon LaBadie wrote: > > >>So... is amanda really not made for disaster recovery? That's great > >>that it will back up to a holding disk with no tape but if the hard > >>drive dies it's rather meaningless. > > > > > >How does that imply "not for disaster recovery"? What would > >your current system do if the tape "jammed"? Would it still > >backup? True, if you encountered a situation where you did > >not put in a tape AND the holding disk died you would not > >have a backup. Is that your concern? > > > > It's because of what I am used to... Right now Arcserve > backs up my Netware server as long as there is a tape in > the drive. To me that's ideal since the data collection > in our factory runs 24-7. Even on Sat and Sun when I'm not > there the tape is still over written with the latest data. > If Amanda does only copies the data to a holding area and > we lose a drive on Sunday night I have massive problems. > That's what I call a disaster and I don't see how amanda > can help me there. I do want to make it work however. > For the disaster you feel is possible several things have to occur at the same time. 1. Your tape or tapedrive fails, or you do not put in a suitable tape - though as has been said by another, with a tapecycle of 1, you could override that protection. 2. A drive containing valuable data would have to fail 3. The drive containing your holding disk would have to fail. Usually this is not the systems primary data drive. And it can also be spread over multiple drives so if one fails others will do the job. What about your current system. Fine, if the wrong tape is inserted you still get a current backup. Same with amanda. It is just on the holding disk. And I'd be worried about something else in your current system. Presently I'm starting to "wear out tapes". Suppose you forgot to change a tape, I'm guessing the tape left in there has the most current backup and is about to be overwritten by the new backup. What if the tape fails, or the backup fails in the middle for some other reason. You will have neither todays nor the last backup. -- Jon H. LaBadie [EMAIL PROTECTED] JG Computing 4455 Province Line Road(609) 252-0159 Princeton, NJ 08540-4322 (609) 683-7220 (fax)
Dump ordering optimisation documentation
Is there any chance we can get this in the documentation? http://marc.theaimsgroup.com/?l=amanda-users&m=106277015010167&w=2
breaking up harddrives for gtar
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Hi, Right now I am using amanda for a few different linux servers. My disklists use the directories where different partitions are mounted. This has been working pretty well, but I read somewhere that one can get even better results by breaking things up to as many small directories as one can. This makes sense to me, as it will allow amanda to schedule different level backups the best she can even if it makes things perhaps more difficult to keep track of. So I collected on each client the names of the harddrives for it. find / -type d -fprintf ~/amandalist "`hostname -f`\t\t%p\tcomp-user-tar\n" and I have an idea to exclude all subdirectories in the dumptype, but then I run into the problem of users creating and removing their own subdirectories and then dealing with those. Do people who advocate this sort of setup generate their disklist each time they run amanda? or do they not have to deal with this problem? What is the best use of gtar? Many thanks, - -- Jason -BEGIN PGP SIGNATURE- Version: GnuPG v1.2.4 (GNU/Linux) iD8DBQFBNOZGA5NiLuECHn4RAkEPAJ4906MoHRzinMz8ye39xx4rHATBtgCgnM/4 ItfI17m2FNhpBvgn2l7nTXg= =l3Gr -END PGP SIGNATURE-
Re: Need help with backup plan
Joe Konecny wrote: Jon LaBadie wrote: So... is amanda really not made for disaster recovery? That's great that it will back up to a holding disk with no tape but if the hard drive dies it's rather meaningless. I don't understand the leap you've made here. The holding disk is a buffer to allow multiple items to be backed up simultaneously but with a single stream to tape. If no "usable" tape is available amanda will still do the backups but leave them in the buffer (holding disk) for later taping. "Usable" here can mean many things including a tape not part of amanda's collection (do you really want it to trash someone else's valuable data?), an amanda tape not permitted to be overwritten (admin controllable), or even a broken drive or no tape inserted. How does that imply "not for disaster recovery"? What would your current system do if the tape "jammed"? Would it still backup? True, if you encountered a situation where you did not put in a tape AND the holding disk died you would not have a backup. Is that your concern? BTW some amanda installations use cheaper, separate drives for the holding disk. On my primary home office system, the 4 system drives are scsi. My holding disk is an IDE drive bigger than the sum of the scsi drives. Once I went on a week long trip forgetting to insert a new set of tapes. Came home, inserted the tapes and flushed a weeks worth of normal backups on to several tapes. It's because of what I am used to... Right now Arcserve backs up my Netware server as long as there is a tape in the drive. To me that's ideal since the data collection in our factory runs 24-7. Even on Sat and Sun when I'm not there the tape is still over written with the latest data. If Amanda does only copies the data to a holding area and we lose a drive on Sunday night I have massive problems. That's what I call a disaster and I don't see how amanda can help me there. I do want to make it work however. I think that if things are so critical I would seriously investigate in purchasing a tape changer.
Re: Need help with backup plan
Jon LaBadie wrote: So... is amanda really not made for disaster recovery? That's great that it will back up to a holding disk with no tape but if the hard drive dies it's rather meaningless. I don't understand the leap you've made here. The holding disk is a buffer to allow multiple items to be backed up simultaneously but with a single stream to tape. If no "usable" tape is available amanda will still do the backups but leave them in the buffer (holding disk) for later taping. "Usable" here can mean many things including a tape not part of amanda's collection (do you really want it to trash someone else's valuable data?), an amanda tape not permitted to be overwritten (admin controllable), or even a broken drive or no tape inserted. How does that imply "not for disaster recovery"? What would your current system do if the tape "jammed"? Would it still backup? True, if you encountered a situation where you did not put in a tape AND the holding disk died you would not have a backup. Is that your concern? BTW some amanda installations use cheaper, separate drives for the holding disk. On my primary home office system, the 4 system drives are scsi. My holding disk is an IDE drive bigger than the sum of the scsi drives. Once I went on a week long trip forgetting to insert a new set of tapes. Came home, inserted the tapes and flushed a weeks worth of normal backups on to several tapes. It's because of what I am used to... Right now Arcserve backs up my Netware server as long as there is a tape in the drive. To me that's ideal since the data collection in our factory runs 24-7. Even on Sat and Sun when I'm not there the tape is still over written with the latest data. If Amanda does only copies the data to a holding area and we lose a drive on Sunday night I have massive problems. That's what I call a disaster and I don't see how amanda can help me there. I do want to make it work however.
Re: Need help with backup plan
Hi, Joe, on Dienstag, 31. August 2004 at 21:49 you wrote to amanda-users: JK> Gavin Henry wrote: >> Amanda is much better than that. It will but the backup in a holding disk >> i.e. folder/partition until space runs out or hits the limit you set. [...] >> Read the docs first and look at this site: >> >> http://www.oops.co.at/AMANDA-docs/index.html Thanks, Gavin. JK> So... is amanda really not made for disaster recovery? That's great JK> that it will back up to a holding disk with no tape but if the hard JK> drive dies it's rather meaningless. Give it a tape. Disaster Recovery is a special thing. AMANDA does still help you in this case, as "she" makes it possible to restore your tapes WITHOUT any AMANDA-software installed. Take any UNIX/Linux with a proper tapedrive and go. It is not a very good argument to say AMANDA is bad, because the disk onto which it dumps your data in the case you failed to change the tape fails. This reminds me of the old problem: You have a light which tells you your brakes failed. What if this light fails? You get a light which tells you that the light failed, that tells you your brakes failed. What if THIS light fails? Dumping to the holdingdisk provides an additional layer of security in the case you put in the wrong tape, your drive eats the tape or something similar. AMANDA is not designed to take care of harddisks. Stefan G. Weichinger mailto:[EMAIL PROTECTED]
Re: Need help with backup plan
Joe Konecny wrote: Gavin Henry wrote: Amanda is much better than that. It will but the backup in a holding disk i.e. folder/partition until space runs out or hits the limit you set. It will not overwrite another tape unless you force it by hand, which is good. You don't have to figure out which tape, amanda tells you. You can even browse the backup files like a normal Unix system and restore per file. You have not spec'd how often you backup either, how much data and much more if we need to help you. Read the docs first and look at this site: http://www.oops.co.at/AMANDA-docs/index.html So... is amanda really not made for disaster recovery? That's great that it will back up to a holding disk with no tape but if the hard drive dies it's rather meaningless. Umm.. You're supposed to flush the holdingdisk to tape as soon as you realise that you forgot to change tapes the other day. Then you are safe..
Re: Need help with backup plan
On Tue, Aug 31, 2004 at 03:49:49PM -0400, Joe Konecny wrote: > Gavin Henry wrote: > > >Amanda is much better than that. It will but the backup in a holding disk > >i.e. folder/partition until space runs out or hits the limit you set. > > > >It will not overwrite another tape unless you force it by hand, which > >is good. > > > >You don't have to figure out which tape, amanda tells you. You can even > >browse the backup files like a normal Unix system and restore per file. > > > >You have not spec'd how often you backup either, how much data and much > >more > >if we need to help you. > > > >Read the docs first and look at this site: > > > >http://www.oops.co.at/AMANDA-docs/index.html > > So... is amanda really not made for disaster recovery? That's great > that it will back up to a holding disk with no tape but if the hard > drive dies it's rather meaningless. I don't understand the leap you've made here. The holding disk is a buffer to allow multiple items to be backed up simultaneously but with a single stream to tape. If no "usable" tape is available amanda will still do the backups but leave them in the buffer (holding disk) for later taping. "Usable" here can mean many things including a tape not part of amanda's collection (do you really want it to trash someone else's valuable data?), an amanda tape not permitted to be overwritten (admin controllable), or even a broken drive or no tape inserted. How does that imply "not for disaster recovery"? What would your current system do if the tape "jammed"? Would it still backup? True, if you encountered a situation where you did not put in a tape AND the holding disk died you would not have a backup. Is that your concern? BTW some amanda installations use cheaper, separate drives for the holding disk. On my primary home office system, the 4 system drives are scsi. My holding disk is an IDE drive bigger than the sum of the scsi drives. Once I went on a week long trip forgetting to insert a new set of tapes. Came home, inserted the tapes and flushed a weeks worth of normal backups on to several tapes. -- Jon H. LaBadie [EMAIL PROTECTED] JG Computing 4455 Province Line Road(609) 252-0159 Princeton, NJ 08540-4322 (609) 683-7220 (fax)
Re: Need help with backup plan
Gavin Henry wrote: Amanda is much better than that. It will but the backup in a holding disk i.e. folder/partition until space runs out or hits the limit you set. It will not overwrite another tape unless you force it by hand, which is good. You don't have to figure out which tape, amanda tells you. You can even browse the backup files like a normal Unix system and restore per file. You have not spec'd how often you backup either, how much data and much more if we need to help you. Read the docs first and look at this site: http://www.oops.co.at/AMANDA-docs/index.html So... is amanda really not made for disaster recovery? That's great that it will back up to a holding disk with no tape but if the hard drive dies it's rather meaningless.
Change ORIG-KB to ORIG-MB or ORIG-GB?
All of my volumes are larger than a megabyte. Is there any to show MB or GB instead of KB values so I don't have to widen my columns so much?
Re: Need help with backup plan
On Tue, Aug 31, 2004 at 10:42:58AM -0400, Joe Konecny wrote: > The system is set up to accept > any tape and run a full backup. What we do is keep the tapes > in rotation and if someone forgets to bring a tape back from > off site it is no big deal we just use the next one. This is the interesting part. Amanda tries to protect you from accidentally overwriting backups that you wanted to keep. To that end, it won't "accept any tape", but only tapes whose contents it considers to have expired. That's what the whole "tapecycle" thing's about. To get what you want, you'll have to defeat this protection. Typically, people have C tapes, and set "tapecycle" to C. Under those conditions (once you've reached equilibrium by running through all the tapes once) Amanda refuses to write to the C-1 most recently used tapes; it will only accept the least recently used tape, C, or a new tape that's never been written to. That's the situation I'm familiar with. But suppose you have more than "tapecycle" tapes, i.e. you have T tapes for some T>C. I believe (please correct me if necessary, folks) that in this case, Amanda will still refuse to write to the C-1 most recently used tapes, but will accept any of the older ones, i.e. any of tapes C through T. It'll say "The next tape Amanda expects to use is: [label for tape T]", but that's not quite true; in fact, it'll accept any tape that isn't in the range [1, C-1]. So, to get the behaviour you have now, I think you can just set "tapecycle" to 1. But I don't know for sure, because I'd never do that :-) Personally, I *like* it that Amanda protects me from my own mistakes (forgetting to change the tape, or mounting the wrong one, for example). Of course, you could use a hybrid approach and get yourself some protection against accidents, but also some of the flexibility you want, simply by setting "tapecycle" to something like 5; then, the latest few tapes would be protected, but any of the older ones would do. > Arcserve > assigns each tape a new ID when it uses it. This, Amanda won't do. You assign each tape an ID (i.e. label) when you amlabel it, and Amanda uses that same label forevermore (unless you manually relabel the tape of course, but Amanda discourages that). That difference in detail shouldn't cause any problem in meeting your basic requirement, though. > We write the date > of the backup on the case of each tape. If a restore is needed > we figure out what date we want and insert the tape. Arcserve > will tell us the tape ID and then we can restore from that > session. In the Amanda world, you wouldn't need to write the backup date on the case. To restore, you'd: - figure out what date you want - use amrecover's "setdate" command, specifying that date - amrecover would tell *you* which tape to mount, asking for it by its label - should you ever switch to doing incremental backups instead of full ones every night, or should your backups grow to more than one tape per run, amrecover might need more than one tape to do a multi-file restore; it'll ask for each one in turn, again by label -- | | /\ |-_|/ > Eric Siegerman, Toronto, Ont.[EMAIL PROTECTED] | | / It must be said that they would have sounded better if the singer wouldn't throw his fellow band members to the ground and toss the drum kit around during songs. - Patrick Lenneau
Interpreting amplot for newbies
For some reason, I though amplot was too difficult to set up and use, so I never tried it for the last three years that I've used Amanda. Now, with a new installation, I've tried it. However, I'm not sure how to read and understand the plots. Would anyone be willing to take a look at the amplot output at http://www.jhuccp.org/20040828.pdf and http://www.jhuccp.org/20040831.pdf and give me any suggestions on how my setup could be tuned? In general, I think everything's okay, since all the lines are below 100% capacity, but that's the limit of what I can get from the plots. Specifically, is it normal or common for there to be an almost 50 minute dead-space at the beginning of each run? I'm familiar with the idea of needing time to compute estimates, but this seems excessive. Also, should I worry about these lines when I produce an amplot: [EMAIL PROTECTED]:/var/amanda/DBackup$ amplot -e -p -l -t 2 amdump.2 Unknown statement# driver: tape size 19714048 20040828 INFO# dumper: bind_portrange: all ports between 850 and 854 busy 20040828 INFO# dumper: bind_portrange: all ports between 10080 and 10083 busy Thanks for your thoughts and suggestions. I appreciate your advice. -Kevin Zembower - E. Kevin Zembower Unix Administrator Johns Hopkins University/Center for Communications Programs 111 Market Place, Suite 310 Baltimore, MD 21202 410-659-6139
RE: Need help with backup plan
Amanda is much better than that. It will but the backup in a holding disk i.e. folder/partition until space runs out or hits the limit you set. It will not overwrite another tape unless you force it by hand, which is good. You don't have to figure out which tape, amanda tells you. You can even browse the backup files like a normal Unix system and restore per file. You have not spec'd how often you backup either, how much data and much more if we need to help you. Read the docs first and look at this site: http://www.oops.co.at/AMANDA-docs/index.html -Original Message- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] Behalf Of Joe Konecny Sent: 31 August 2004 15:43 To: [EMAIL PROTECTED] Subject: Need help with backup plan Right now I back up my Netware server with Arcserve to a single tape. I have 10 tapes. The system is set up to accept any tape and run a full backup. What we do is keep the tapes in rotation and if someone forgets to bring a tape back from off site it is no big deal we just use the next one. Arcserve assigns each tape a new ID when it uses it. We write the date of the backup on the case of each tape. If a restore is needed we figure out what date we want and insert the tape. Arcserve will tell us the tape ID and then we can restore from that session. Now I'm switching to FreeBSD and have a working Amanda install with 10 tapes. I have yet to configure the cycle, etc... as I'm not sure how to make it work like Arcserve did. Can Amanda be made to work like that?
Re: Need help with backup plan
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On Tuesday, 31.08.2004 at 10:42 -0400, Joe Konecny wrote: > Right now I back up my Netware server with Arcserve to a single tape. > I have 10 tapes. The system is set up to accept any tape and run a > full backup. What we do is keep the tapes in rotation and if someone > forgets to bring a tape back from off site it is no big deal we just > use the next one. Arcserve assigns each tape a new ID when it uses > it. We write the date of the backup on the case of each tape. If a > restore is needed we figure out what date we want and insert the tape. > Arcserve will tell us the tape ID and then we can restore from that > session. Now I'm switching to FreeBSD and have a working Amanda > install with 10 tapes. I have yet to configure the cycle, etc... as > I'm not sure how to make it work like Arcserve did. Can Amanda be > made to work like that? Won't comment specifically for your case but most questions of "Can Amanda be made to work like that?" are normally not the right thing to ask. Find out how AMANDA actually *does* it and then fit in with its way of doing things. AMANDA is good at scheduling backups. Dave. - -- Dave Ewart [EMAIL PROTECTED] Computing Manager, Epidemiology Unit, Oxford Cancer Research UK PGP: CC70 1883 BD92 E665 B840 118B 6E94 2CFD 694D E370 -BEGIN PGP SIGNATURE- Version: GnuPG v1.2.4 (GNU/Linux) iD8DBQFBNI/fbpQs/WlN43ARAlVzAKC70NcgufKVQtbeK4DLGacf0AMzYgCaA8Gi xZjFI6PMHi4RKbfyZMmf19g= =Qo1J -END PGP SIGNATURE-
Need help with backup plan
Right now I back up my Netware server with Arcserve to a single tape. I have 10 tapes. The system is set up to accept any tape and run a full backup. What we do is keep the tapes in rotation and if someone forgets to bring a tape back from off site it is no big deal we just use the next one. Arcserve assigns each tape a new ID when it uses it. We write the date of the backup on the case of each tape. If a restore is needed we figure out what date we want and insert the tape. Arcserve will tell us the tape ID and then we can restore from that session. Now I'm switching to FreeBSD and have a working Amanda install with 10 tapes. I have yet to configure the cycle, etc... as I'm not sure how to make it work like Arcserve did. Can Amanda be made to work like that?
Out-of-office replies?
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Dear AMANDA users, Recent posts to the AMANDA users list have attracted a lot of "out of office" replies. I also used to send "please don't autoreply to mailing list" messages back to these subscribers. However, looking at the headers of a typical amanda-users post, there is *nothing* there to help autoreply/vacation software identify it as a mailing list post. Can some list admin please add "Precedence: list" or something to the message headers? Dave. - -- Dave Ewart [EMAIL PROTECTED] Computing Manager, Epidemiology Unit, Oxford Cancer Research UK PGP: CC70 1883 BD92 E665 B840 118B 6E94 2CFD 694D E370 -BEGIN PGP SIGNATURE- Version: GnuPG v1.2.4 (GNU/Linux) iD8DBQFBNICwbpQs/WlN43ARAnrrAJoDlUyxFafCmKGvxSi0B1aKnuZ9+gCeMweD gz7j+eMdSGjVqpKlZRFAYho= =akj7 -END PGP SIGNATURE-
Re: How do you stop Amanda overwriting a tape?
Errm, Therre seems to be something broken in your weekly and Pedcent configuration. the regexpressions giving the labels for "weekly" and "Pedcent" should be choosen in a way that "Pedcent" tapes are not accepted by "weekly" and vice versa. And make shure the two configs don't share the tapelist. Then you get exactly the behavior you want. i would suggest putting something like labelstring "Peg[0-9].[0-9]*" in the amanda.conf of your Pedcent-config and labelstring "Fri[0-9].[0-9]*" in the weekly amanda.conf. I use labelstring "Weekly[0-9]*" for my weekly-tapes, but thats my way of doing this Christoph Ranveer Attalia schrieb: I schedule a full backup of our servers on Fridays which is part of the "Weekly" amanda config. I also schedule a separate full backup of our critical server called Pedcent on Sundays which is part of the "Pedcent" amanda config: $ crontab -l 00 22 * * 1-4 /opt/amanda/sbin/ambackup Daily >> /tmp/ambackup_Daily.log 2>&1 00 21 * * 5 /opt/amanda/sbin/ambackup Weekly >> /tmp/ambackup_Weekly.log 2>&1 00 14 * * 0 /opt/amanda/sbin/ambackup Pedcent >> /tmp/ambackup_Pedcent.log 2>&1 Hence the tapes I put into the jukebox on Fridays are as follows: Slot 1: Fri3.1 Slot 2: Fri3.2 Slot 3: Peg0.1 where Peg0.1 is used for the "Pedcent" full backup. Evey Monday I receive 2 emails. One telling me that the "Weekly" was successful and the other telling me that "Pedcent" was successful. Here is where the BUT comes in.. After doing a recent system disk restore I noticed that when searching for a server to restore from the "Weekly" disklist, it told me to load the Fri3.1 tape. When I read it however, it was displaying all of "Pedcent" partitions. This means that it had actually overwritten the first tape. Did this happen because there wasnt enough room on the Peg0.1 tape so it just went to Fri3.1 ? (there is plenty of room on the Peg0.1 tape by the way) Is there a way that Amanda will request a new tape to be loaded if it reaches to the end of its assigned tape, without going off and overwriting a previous one? Thanks - Ranveer Tertio Telecoms (www.telco-tertio.com) Head Office: One Angel Square Torrens Street London EC1V 1PL Tel: +44 (0)20 7843 4000 - Fax: +44 (0)20 7843 4001 Bath Office: Riverside Buildings 108 Walcot Street Bath BA1 5BG Tel: +44 (0)1225 478000 - Fax: +44 (0)1225 478001 Munich Office: Freisinger Strasse 10, 85737 Ismaning/Munich, Germany, Tel: +49 (0)89 665506 41 - Fax: +49 (0)89 665506 99 Any views expressed in this message are those of the individual sender, except where the sender specifically states them to be the views of Tertio Ltd. This email has been scanned by the MessageLabs Email Security System.
Re: Labeling tapes with "day of the week", is not a good practice
On Tue, Aug 31, 2004 at 01:55:23PM +0100, Ranveer Attalia wrote: > > Hi > > First thanks to everyone that helped me with my nightmare restore last > week. I've managed to get the system disk up and running now so things > are little calmer. Trying to understand the errors I recieved during the > restore and (with the ... > system disk restore, I eventually managed to get all the data off) > Glad you were able to get back running. > Gavin mentioned that we shouldnt be using "day of the week" > tape labels. Why is this? Are the results fatal if we do? The labels serve two masters, you and amanda. Your label is probably on the outside of the tape, amanda's is on the tape itself. So actually they don't even have to match. Its a mess though if they don't. :( Amanda wants her tapes in order, doesn't matter about the label. The order could be 1,2,3 or 293,482,111 or mon,tue,wed, or huey,dewey,louie. Doesn't matter to amanda. As long as she sees them in the same order the next time. You on the other hand think a tape labeled 3 must follow a tape labeled 2 (or tuesday must follow monday). And you may feel that a tape labeled tuesday must be used on tuesday. Silly you. :)) It has been the collective experience of amanda users that any scheme to order the tapes for your purposes (not amanda's) will get out of order. Tapes will be added, go bad and be replaced. Computers will fail and a dump will be missed or someone will forget to change the tapes or a dump that normally takes 3 tapes one day will take 2 or 4. So generally it is recommended that you keep your labeling simple and don't concern yourself if eventually they don't follow a human-based logical order. > > All of our Daily and Weekly tapes are labelled with the day of the week. > Is there any easy way that we can revert back to the Amanda way of > labelling the tapes? If it is possible and we can revert, would we still > beable to restore off the "day of the week" tapes that we have up until > now if we ever need to? If you can mentally deal with using tuesdays tape on friday, they don't "have to be" relabelled. One detail-intensive way to redo them is to relabel them as they are about to be reused. Use the "-f" option of amlabel to overwrite the current label. Amanda will consider the tape to be a "new" tape and it will always use a new tape even if it is beyond the number of tapes in the tapecycle. To enforce this, you could mark each tape as "no-reuse". Then amanda will require a "new" tape each run. Someone correct me if I am wrong, but I believe this will preserve the index for those tapes. Of course, after relabeling and reusing the tape you will want to remove the tape from the database rather than just listing it as "no-reuse" (amrmtape). HTH -- Jon H. LaBadie [EMAIL PROTECTED] JG Computing 4455 Province Line Road(609) 252-0159 Princeton, NJ 08540-4322 (609) 683-7220 (fax)
Re: Labeling tapes with "day of the week", is not a good practice
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On Tuesday, 31.08.2004 at 13:55 +0100, Ranveer Attalia wrote: > All of our Daily and Weekly tapes are labelled with the day of the > week. Is there any easy way that we can revert back to the Amanda way > of labelling the tapes? If it is possible and we can revert, would we > still beable to restore off the "day of the week" tapes that we have > up until now if we ever need to? You can migrate your currently named tapes from your 'daily' names to some other, 'non-daily' names. I have done this! You do this by gradually changing the names of tapes as you go through your normal backup cycle. We have 20 tapes and it took 20 days to make all these changes. For instance, say you want to change from Blah-Monday-1, Blah-Tuesday-1 etc. to Blah-001, Blah-002 etc. You do this: 1. Change the regular expression in amanda.conf to recognize *both* the old and the new tape names. This is the 'trick' to making this work. 2. When a tape is due for re-use, just before use, you relabel it according to the new scheme. You then 'amrmtape' the old name. 3. Do this each day for your entire tapecycle. After $TAPECYCLE days, all your tapes will be labelled with the new scheme and you can change the label regexp in amanda.conf to just recognize the new labels. At all times, you are able to restore for any tape, as normal, whether named new or old. Dave. - -- Dave Ewart [EMAIL PROTECTED] Computing Manager, Epidemiology Unit, Oxford Cancer Research UK PGP: CC70 1883 BD92 E665 B840 118B 6E94 2CFD 694D E370 -BEGIN PGP SIGNATURE- Version: GnuPG v1.2.4 (GNU/Linux) iD8DBQFBNHk0bpQs/WlN43ARAgJZAJ9m4HIzVqJ3bD/zva5E9Yy8MpxS+gCg8Yyy 25RThkVU8KD6sVDe/FRvZAc= =0DDO -END PGP SIGNATURE-
How do you stop Amanda overwriting a tape?
I schedule a full backup of our servers on Fridays which is part of the "Weekly" amanda config. I also schedule a separate full backup of our critical server called Pedcent on Sundays which is part of the "Pedcent" amanda config: $ crontab -l00 22 * * 1-4 /opt/amanda/sbin/ambackup Daily >> /tmp/ambackup_Daily.log 2>&100 21 * * 5 /opt/amanda/sbin/ambackup Weekly >> /tmp/ambackup_Weekly.log 2>&100 14 * * 0 /opt/amanda/sbin/ambackup Pedcent >> /tmp/ambackup_Pedcent.log 2>&1 Hence the tapes I put into the jukebox on Fridays are as follows: Slot 1: Fri3.1 Slot 2: Fri3.2 Slot 3: Peg0.1 where Peg0.1 is used for the "Pedcent" full backup. Evey Monday I receive 2 emails. One telling me that the "Weekly" was successful and the other telling me that "Pedcent" was successful. Here is where the BUT comes in.. After doing a recent system disk restore I noticed that when searching for a server to restore from the "Weekly" disklist, it told me to load the Fri3.1 tape. When I read it however, it was displaying all of "Pedcent" partitions. This means that it had actually overwritten the first tape. Did this happen because there wasnt enough room on the Peg0.1 tape so it just went to Fri3.1 ? (there is plenty of room on the Peg0.1 tape by the way) Is there a way that Amanda will request a new tape to be loaded if it reaches to the end of its assigned tape, without going off and overwriting a previous one? Thanks - Ranveer Tertio Telecoms (www.telco-tertio.com) Head Office: One Angel Square Torrens Street London EC1V 1PL Tel: +44 (0)20 7843 4000 - Fax: +44 (0)20 7843 4001 Bath Office: Riverside Buildings 108 Walcot Street Bath BA1 5BG Tel: +44 (0)1225 478000 - Fax: +44 (0)1225 478001 Munich Office: Freisinger Strasse 10, 85737 Ismaning/Munich, Germany, Tel: +49 (0)89 665506 41 - Fax: +49 (0)89 665506 99 Any views expressed in this message are those of the individual sender, except where the sender specifically states them to be the views of Tertio Ltd. This email has been scanned by the MessageLabs Email Security System.
Labeling tapes with "day of the week", is not a good practice
Hi First thanks to everyone that helped me with my nightmare restore last week. I've managed to get the system disk up and running now so things are little calmer. Trying to understand the errors I recieved during the restore and Gavin mentioned that we shouldnt be using "day of the week" tape labels. Why is this? Are the results fatal if we do? (with the system disk restore, I eventually managed to get all the data off) All of our Daily and Weekly tapes are labelled with the day of the week. Is there any easy way that we can revert back to the Amanda way of labelling the tapes? If it is possible and we can revert, would we still beable to restore off the "day of the week" tapes that we have up until now if we ever need to? Thanks - Ranveer -Original Message- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] Behalf Of Paul Bijnens Sent: 24 August 2004 14:51 To: Ranveer Attalia Cc: [EMAIL PROTECTED] Subject: Re: Anyone please reply! System Disk recovery - UNKNOWN file. amrecover: Cant read file header Ranveer Attalia wrote: > I am trying to test restore before I run a system disk recover from > amanda server to my client machine. > The restore file is any one from /opt onto the client telhp6 onto a > /scratch/restored directory but its giving me file header errors... > I cannot understand why though because it looks as if it backed up > fine and its displaying the tape labels with no problem on the amanda server: > Please can someone help > > Thanks > > - Ranveer > > $ amtape Weekly show > amtape: scanning all 7 slots in tape-changer rack: > slot 1: date 20040822 label Fri3.1 > slot 2: date 20040820 label Fri3.2 > amtape: could not load slot source: Element Address 8 is Empty Labeling tapes with "day of the week", is not a good practice, just in case you didn't read the earlier warnings about this issue. Tertio Telecoms (www.telco-tertio.com) Head Office: One Angel Square Torrens Street London EC1V 1PL Tel: +44 (0)20 7843 4000 - Fax: +44 (0)20 7843 4001 Bath Office: Riverside Buildings 108 Walcot Street Bath BA1 5BG Tel: +44 (0)1225 478000 - Fax: +44 (0)1225 478001 Munich Office: Freisinger Strasse 10, 85737 Ismaning/Munich, Germany, Tel: +49 (0)89 665506 41 - Fax: +49 (0)89 665506 99 Any views expressed in this message are those of the individual sender, except where the sender specifically states them to be the views of Tertio Ltd. This email has been scanned by the MessageLabs Email Security System.
