Re: Requesting comments on a possible amanda configuration
"Carl D. Blake" <[EMAIL PROTECTED]> wrote: >I would prefer to let amanda handle the scheduling. I'm just having >trouble figuring out a reasonable configuration where I can get weekly >full backups as well as monthly (or every 4 weeks) full backups that I >can archive for a year. > >You mention that your archive config runs once a week and does monthly >backups for each filesystem. Is that configuration something like: > dumpcycle 0 > runspercycle 4 > tapecycle 52 >where dumpcycle 0 tells amanda to perform full backups every time, >runspercycle 4 tells amanda to put the full backups on 4 tapes, and >tapecycle 52 provides enough tapes for a year? Actually, the archival config is more like: dumpcycle4 weeks# 1 full backup / month runspercycle 4 # 12 dailys + 2 archives / fortnight tapecycle1000 tapes # the number of tapes in rotation (We don't recycle the archival tapes. It may be time to bump up the 1000, come to think of it.) The dumptypes have: record no skip-incr yes The daily config is something like: dumpcycle 2 weeks # 1 full backup / fortnight runspercycle 12 # 12 dailys + 2 archives / fortnight tapecycle20 tapes # the number of tapes in rotation >Also, how many tapes do you use for your daily config? Is it enough to >cover an entire month? 20, which isn't quite enough. Since most of the I-deleted-something-I-shouldn't-have restores are handled by the Network Appliance's snapshots, our backups are mostly for minor disaster recovery (disk failure and the like) and we only need tapecycle > runspercycle. Tommy McGuire
Re: Requesting comments on a possible amanda configuration
On Tue, 2002-11-12 at 14:43, [EMAIL PROTECTED] wrote: > > > "Carl D. Blake" <[EMAIL PROTECTED]> wrote: > >On Tue, 2002-11-12 at 10:56, [EMAIL PROTECTED] wrote: > >> > >> > >> "Carl D. Blake" <[EMAIL PROTECTED]> wrote: > >> >I'm trying to setup an AMANDA backup system with the following > >> >characteristics: > >> >Full backup every Wednesday on a different weekly tape > >> >Differential backup on Thurday, Friday, Monday, and Tuesday > >> >Full backup on the last Wednesday of each month which is then > >> >archived for a year. > >> > >> Comments or suggestions? You are trying to swim upstream. > >> > >> Tommy McGuire > >Could you explain what you mean by this? What is a better way to handle > >this? I would prefer to use amanda the way it was meant to be used > >rather than try to force her into some peculiar mode. > > Amanda is really very good at set-it-and-forget-it backups: you tell it > what you want to back up, how often you want a full backup, and how many > tapes to use. It figures out when to do what level of backup to spread > out your backups across the schedule. > > For example, we have two configurations. The daily config, which is run > 6 nights per week, is set to do a full backup of each filesystem every > two weeks, incrementals in between, and to update the equivalent of > /etc/dumpdates with when it does what level. The archive config, which > runs once a week, doesn't do incremental backups, doesn't record what > it does, and does a full backup of each filesystem once a month. This > configuration is very similar to what is described above except that > it doesn't mention what day of the week or month things are supposed to > happen. > This sounds similar to what I want to do with our backups. I've been trying to get used to the idea of not knowing when a full backup or an incremental backup occurs. This is why I was asking for comments and suggestions. > Do I know what level the daily config is doing for a particular > filesystem on a given day? Not really. Do I know what week a > particular filesystem will get archived? Nope. Do I care? No. When > we have to restore something, Amanda tells us what tapes to get it off > of. What does it get us? Back before Amanda, when we had a schedule > somewhat like that above, the archive run once a month was taking > between two and three days. Some of the dailys wanted several tapes > and others less than half of one. This was a while back; that kind of > config wouldn't work for us at all now. > > When you try to tell Amanda to do a full backup every Wednesday, or to > do a full backup of everything the last Wednesday of the month, you are > trying to manually do something that Amanda was designed to automate. > You're not using Amanda's power and you're going have to do perverted > things to the configuration to get it to work at all, which it probably > won't. You are swimming upstream, and it would be easier to write your > own backup scripts to follow your schedule. If you need Amanda's power, > you have to let it handle the scheduling. > I would prefer to let amanda handle the scheduling. I'm just having trouble figuring out a reasonable configuration where I can get weekly full backups as well as monthly (or every 4 weeks) full backups that I can archive for a year. You mention that your archive config runs once a week and does monthly backups for each filesystem. Is that configuration something like: dumpcycle 0 runspercycle 4 tapecycle 52 where dumpcycle 0 tells amanda to perform full backups every time, runspercycle 4 tells amanda to put the full backups on 4 tapes, and tapecycle 52 provides enough tapes for a year? Also, how many tapes do you use for your daily config? Is it enough to cover an entire month? > Tommy McGuire -- Carl D. Blake Director of Engineering Boeckeler Instruments, Inc. 4650 S. Butterfield Dr. Tucson, AZ 85714 Phone: 520-745-0001 FAX: 520-745-0004 email: [EMAIL PROTECTED] .com
Re: Requesting comments on a possible amanda configuration
On Tue, Nov 12, 2002 at 01:36:54PM -0700, Carl D. Blake wrote: > On Tue, 2002-11-12 at 11:59, Jay Lessert wrote: [clip] > > config "daily", run 5X/week: [clip] > > config "monthly", run 1X/month [clip] > This doesn't sound too bad. My only question is how would I run the > monthly config. Would I just pick a day in the month on which I would > run "amdump monthly" instead of "amdump daily"? Do what you like, that's *one* choice Amanda won't make for you. :-) If I were you, and since (apparently) you want to run your dailies 5X/week on Monday-Friday, I would run my monthly on the first Saturday of each month, something like: % sudo crontab -l amanda # Run daily on mon/tue/wed/thu/fri nights (0=sunday) 01 23 * * 1-5 /home/amanda/bin/daily # # Run monthly on every sat night, it exits if DOM > 7 01 23 * * 6 /home/amanda/bin/monthly ~amanda/bin/monthly: #!/bin/sh if expr `date +%d` \< 8 ; then amdump monthly amtape monthly eject else exit 0 fi > If so, would that cause > the daily dumps to get offset by one day? Does this even matter? Note that the "monthly" config runs with 'record no', so the "daily" config has no idea that "monthly" exists. If for some reason you can't run "monthly" on an off night (no changer, for example), then yes, you would have to arrange for your cron/script environment to either: - Run "daily", then "monthly" in series 1 day/month - Skip "daily", run "monthly" instead 1 day/month. Best is a changer with one slot dedicated to the "monthly" config, though. Make that happen if you possibly can. -Jay-
Re: Requesting comments on a possible amanda configuration
"Carl D. Blake" <[EMAIL PROTECTED]> wrote: >On Tue, 2002-11-12 at 10:56, [EMAIL PROTECTED] wrote: >> >> >> "Carl D. Blake" <[EMAIL PROTECTED]> wrote: >> >I'm trying to setup an AMANDA backup system with the following >> >characteristics: >> > Full backup every Wednesday on a different weekly tape >> > Differential backup on Thurday, Friday, Monday, and Tuesday >> > Full backup on the last Wednesday of each month which is then >> > archived for a year. >> >> Comments or suggestions? You are trying to swim upstream. >> >> Tommy McGuire >Could you explain what you mean by this? What is a better way to handle >this? I would prefer to use amanda the way it was meant to be used >rather than try to force her into some peculiar mode. Amanda is really very good at set-it-and-forget-it backups: you tell it what you want to back up, how often you want a full backup, and how many tapes to use. It figures out when to do what level of backup to spread out your backups across the schedule. For example, we have two configurations. The daily config, which is run 6 nights per week, is set to do a full backup of each filesystem every two weeks, incrementals in between, and to update the equivalent of /etc/dumpdates with when it does what level. The archive config, which runs once a week, doesn't do incremental backups, doesn't record what it does, and does a full backup of each filesystem once a month. This configuration is very similar to what is described above except that it doesn't mention what day of the week or month things are supposed to happen. Do I know what level the daily config is doing for a particular filesystem on a given day? Not really. Do I know what week a particular filesystem will get archived? Nope. Do I care? No. When we have to restore something, Amanda tells us what tapes to get it off of. What does it get us? Back before Amanda, when we had a schedule somewhat like that above, the archive run once a month was taking between two and three days. Some of the dailys wanted several tapes and others less than half of one. This was a while back; that kind of config wouldn't work for us at all now. When you try to tell Amanda to do a full backup every Wednesday, or to do a full backup of everything the last Wednesday of the month, you are trying to manually do something that Amanda was designed to automate. You're not using Amanda's power and you're going have to do perverted things to the configuration to get it to work at all, which it probably won't. You are swimming upstream, and it would be easier to write your own backup scripts to follow your schedule. If you need Amanda's power, you have to let it handle the scheduling. Tommy McGuire
Re: Requesting comments on a possible amanda configuration
On Tue, 2002-11-12 at 11:28, Jon LaBadie wrote: > BTW I forgot, > amand will not do an incremental until a level 0 is done for that config. > So I don't think your dailies could be just incrementals. Again, maybe, > just maybe, with merged info/index files. > That's interesting because I've been doing daily backups and they seem to be working just fine as difference backups. Of course, the fact that I set strategy to nofull in the global dumptype for the Daily config might have something to do with this. -- Carl D. Blake Director of Engineering Boeckeler Instruments, Inc. 4650 S. Butterfield Dr. Tucson, AZ 85714 Phone: 520-745-0001 FAX: 520-745-0004 email: [EMAIL PROTECTED] .com
Re: Requesting comments on a possible amanda configuration
On Tue, 2002-11-12 at 10:56, [EMAIL PROTECTED] wrote: > > > "Carl D. Blake" <[EMAIL PROTECTED]> wrote: > >I'm trying to setup an AMANDA backup system with the following > >characteristics: > >Full backup every Wednesday on a different weekly tape > >Differential backup on Thurday, Friday, Monday, and Tuesday > >Full backup on the last Wednesday of each month which is then > >archived for a year. > > > >So far I've setup a Weekly configuration with runspercycle set to 0, > >dumpcycle set to 0, and tapecycle set to 7. record is set to yes so > >that differentials run on a daily basis will have a date that the last > >full backup was done on. > > > >I have a Monthly configuration with runspercycle set to 0, dumpcycle set > >to 0, and tapecycle set to 14. Again, record is set to yes for the sake > >of the daily differentials. > > > >The Daily configuration is runspercycle set to 4, dumpcycle set to 1 > >weeks, and tapecycle set to 7. > > > >I have cron setup to run "amdump Daily" on Monday, Tuesday, Thursday, > >and Friday; "amdump Weekly" on Wednesday for the first 3 (or 4) weeks of > >the month; and "amdump Monthly" on the last Wednesday of the month. > > > >Does this sound like my configuration will do what I want? Or do I have > >something really messed up? Any comments or suggestions would be > >appreciated. > > Comments or suggestions? You are trying to swim upstream. > > Tommy McGuire Could you explain what you mean by this? What is a better way to handle this? I would prefer to use amanda the way it was meant to be used rather than try to force her into some peculiar mode. -- Carl D. Blake Director of Engineering Boeckeler Instruments, Inc. 4650 S. Butterfield Dr. Tucson, AZ 85714 Phone: 520-745-0001 FAX: 520-745-0004 email: [EMAIL PROTECTED] .com
Re: Requesting comments on a possible amanda configuration
On Tue, 2002-11-12 at 11:59, Jay Lessert wrote: > On Tue, Nov 12, 2002 at 10:29:08AM -0700, Carl D. Blake wrote: > [clip] > > this kind of arrangement you can restore within a day for the previous > > week, within a week for the previous month, and within a month for the > > previous year. > > Carl, > > If this is your goal, and you want to use Amanda, I would suggest > this: > > config "daily", run 5X/week: > - dumpcycle 7 days > - runspercycle 5 > - tapecycle 25 > - index yes > - record yes > - strategy normal > > config "monthly", run 1X/month > - dumpcycle 0 > - runspercycle 1 > - tapecycle 12 > - index yes > - record no > - strategy noinc > > This gets you within a day for 5 weeks, and within a month for 1 year, > using 37 tapes. It will "just work", with very little effort on your > part, because there are Amanda users all over the world doing something > *very* close this. > > Note that I've freely mixed dumptype and non-dumptype parameters here > for brevity, you'll be more careful, of course. :-) > > Once you get over the extra tapes (hey, 37 > 20, but << 80!) and get > used to the idea that Amanda will do your daily fulls when *she* wants > (not when you want), you'll like it. Trust me. :-) > This doesn't sound too bad. My only question is how would I run the monthly config. Would I just pick a day in the month on which I would run "amdump monthly" instead of "amdump daily"? If so, would that cause the daily dumps to get offset by one day? Does this even matter? Thanks for your help. -- Carl D. Blake Director of Engineering Boeckeler Instruments, Inc. 4650 S. Butterfield Dr. Tucson, AZ 85714 Phone: 520-745-0001 FAX: 520-745-0004 email: [EMAIL PROTECTED] .com
Re: Requesting comments on a possible amanda configuration
On Tue, Nov 12, 2002 at 10:29:08AM -0700, Carl D. Blake wrote: [clip] > this kind of arrangement you can restore within a day for the previous > week, within a week for the previous month, and within a month for the > previous year. Carl, If this is your goal, and you want to use Amanda, I would suggest this: config "daily", run 5X/week: - dumpcycle 7 days - runspercycle 5 - tapecycle 25 - index yes - record yes - strategy normal config "monthly", run 1X/month - dumpcycle 0 - runspercycle 1 - tapecycle 12 - index yes - record no - strategy noinc This gets you within a day for 5 weeks, and within a month for 1 year, using 37 tapes. It will "just work", with very little effort on your part, because there are Amanda users all over the world doing something *very* close this. Note that I've freely mixed dumptype and non-dumptype parameters here for brevity, you'll be more careful, of course. :-) Once you get over the extra tapes (hey, 37 > 20, but << 80!) and get used to the idea that Amanda will do your daily fulls when *she* wants (not when you want), you'll like it. Trust me. :-) -- Jay Lessert [EMAIL PROTECTED] Accelerant Networks Inc. (voice)1.503.439.3461 Beaverton OR, USA(fax)1.503.466.9472
Re: Requesting comments on a possible amanda configuration
On Tue, Nov 12, 2002 at 10:29:08AM -0700, Carl D. Blake wrote: > On Fri, 2002-11-08 at 10:29, Jon LaBadie wrote: > > On Fri, Nov 08, 2002 at 09:54:46AM -0700, Carl D. Blake wrote: > > > > This is an "i don't know, but consider this" type of comment. > > > > How would you do a recovery? > > > > amrecover requires a config and your Daily config knows nothing > > about the level 0 backup except their dates. It seems to me that > > it will not know what tape you need to load to get the level 0 > > starting point. > > > This is a good point. It looks like I would have to "know" which config > to use to get a particular file. In other words I would have to know > that a file changed since the last full backup and use the last Daily > tape rather than getting the file from the last full backup. However, > for a complete disaster recovery you almost always restore the last full > backup tape first and then do the last difference (level 1) tape. BTW I forgot, amand will not do an incremental until a level 0 is done for that config. So I don't think your dailies could be just incrementals. Again, maybe, just maybe, with merged info/index files. -- Jon H. LaBadie [EMAIL PROTECTED] JG Computing 4455 Province Line Road(609) 252-0159 Princeton, NJ 08540-4322 (609) 683-7220 (fax)
Re: Requesting comments on a possible amanda configuration
"Carl D. Blake" <[EMAIL PROTECTED]> wrote: >I'm trying to setup an AMANDA backup system with the following >characteristics: > Full backup every Wednesday on a different weekly tape > Differential backup on Thurday, Friday, Monday, and Tuesday > Full backup on the last Wednesday of each month which is then > archived for a year. > >So far I've setup a Weekly configuration with runspercycle set to 0, >dumpcycle set to 0, and tapecycle set to 7. record is set to yes so >that differentials run on a daily basis will have a date that the last >full backup was done on. > >I have a Monthly configuration with runspercycle set to 0, dumpcycle set >to 0, and tapecycle set to 14. Again, record is set to yes for the sake >of the daily differentials. > >The Daily configuration is runspercycle set to 4, dumpcycle set to 1 >weeks, and tapecycle set to 7. > >I have cron setup to run "amdump Daily" on Monday, Tuesday, Thursday, >and Friday; "amdump Weekly" on Wednesday for the first 3 (or 4) weeks of >the month; and "amdump Monthly" on the last Wednesday of the month. > >Does this sound like my configuration will do what I want? Or do I have >something really messed up? Any comments or suggestions would be >appreciated. Comments or suggestions? You are trying to swim upstream. Tommy McGuire
Re: Requesting comments on a possible amanda configuration
On Fri, 2002-11-08 at 10:29, Jon LaBadie wrote: > On Fri, Nov 08, 2002 at 09:54:46AM -0700, Carl D. Blake wrote: > > I'm trying to setup an AMANDA backup system with the following > > characteristics: > > Full backup every Wednesday on a different weekly tape > > Differential backup on Thurday, Friday, Monday, and Tuesday > > Full backup on the last Wednesday of each month which is then > > archived for a year. > > > > So far I've setup a Weekly configuration with runspercycle set to 0, > > dumpcycle set to 0, and tapecycle set to 7. record is set to yes so > > that differentials run on a daily basis will have a date that the last > > full backup was done on. > > > > I have a Monthly configuration with runspercycle set to 0, dumpcycle set > > to 0, and tapecycle set to 14. Again, record is set to yes for the sake > > of the daily differentials. > > > > The Daily configuration is runspercycle set to 4, dumpcycle set to 1 > > weeks, and tapecycle set to 7. > > > > I have cron setup to run "amdump Daily" on Monday, Tuesday, Thursday, > > and Friday; "amdump Weekly" on Wednesday for the first 3 (or 4) weeks of > > the month; and "amdump Monthly" on the last Wednesday of the month. > > > > Does this sound like my configuration will do what I want? Or do I have > > something really messed up? Any comments or suggestions would be > > appreciated. > > > This is an "i don't know, but consider this" type of comment. > > How would you do a recovery? > > amrecover requires a config and your Daily config knows nothing > about the level 0 backup except their dates. It seems to me that > it will not know what tape you need to load to get the level 0 > starting point. > This is a good point. It looks like I would have to "know" which config to use to get a particular file. In other words I would have to know that a file changed since the last full backup and use the last Daily tape rather than getting the file from the last full backup. However, for a complete disaster recovery you almost always restore the last full backup tape first and then do the last difference (level 1) tape. > Maybe, just maybe, you could have common curinfo and index directories > among the three configs, and common disklists (linked) as well. > Interesting idea. I'll have to look at this. I'm a little afraid that I'll mess up the curinfo and index databases. The disklists are linked between the three configs right now. > I have multiple archive configs, one for system dirs and one for user > data dirs. They share curinfo and index dirs, but different disklist > files and never do incrementals. > > Separate topic, how do you have your crontab file set up for Weekly > and Monthly? I know how to set it up for the first Wed and all Wed > but the first one, but I'm not sure how to do the last Wed and all > Wed but the last one. I would end up running every Wed a shell > script that selects the correct amdump command. > I setup my crontab file to run a script called "runamdump" every day of the week. The crontab file runs the script as "runamdump Weekly" for the first 24 days of 31 day months, the first 23 days of 30 day months, and the first 21 days of February (this scheme doesn't handle leap years yet). The crontab file runs the script as "runamdump Monthly" for the days 25-31 for 31 day months, 24-30 for 30 day months and 22-28 for February. The script file checks to see if the current day is Wednesday by using the "date" command and if it is then runs amdump, otherwise it exits. Rather complicated, but it seems to work so far. However, I've only been doing this for about a week. I'm sure that there is some debugging in my future. The reason I'm trying to setup my backup system in this manner is because I want to have 3 to 4 sets of Weekly backups and 12 sets of Monthly backups as well as the normal daily difference backups. With this kind of arrangement you can restore within a day for the previous week, within a week for the previous month, and within a month for the previous year. With amanda I couldn't see how to do this without a huge number of tapes in it's normal configuration. I would need 5 tapes for each week (15 to 20) and 5 tapes for each month (60) for a grand total of 80 tapes. Currently a complete full backup of all our systems being backed up using amanda will fit on 1 tape. This means the total number of tapes I actually need are 4 tapes for the daily differences, 3 to 4 tapes for the weekly backups, and 12 tapes for the monthly backups for a grand total of 20 tapes. This is a lot fewer tapes (obviously :) and a lot easier to deal with. Any other suggestions or comments? > -- > Jon H. LaBadie [EMAIL PROTECTED] > JG Computing > 4455 Province Line Road(609) 252-0159 > Princeton, NJ 08540-4322 (609) 683-7220 (fax) -- Carl D. Blake Director of Engineering Boeckeler Instruments, Inc. 4650 S. Butterfield Dr. Tucson, AZ 85714 Phone: 52
