Re: special backupcase
On Thursday 05 September 2002 11:48, Gene Heskett wrote: >On Thursday 05 September 2002 09:29, Thomas Kirk wrote: >>Hey Jim >> >>On Thu, Sep 05, 2002 at 08:01:52AM -0500, Jim Summers wrote: >> >>First of all thanks for the quick answer! :) >> >>> I think you will have to set up multiple disklist / configs to >>> try and force Amanda to do this. >> >>Hmm this seems abit complicated to me as im still trying to grasp >> the basic concepts of amanda? If you could point me towards some >> reading or howto i would be very grateful! >> >>> Amanda herself determines the level and when that level will be >>> done within the dumpcycle. She tries to balance the amount of >>> data backed up each run so as to maximize tape usage. > >Actually, its to equalize tape usage. :-) > >>So what you are telling me is that i can't control wich days to >> do the full backup and which days to do incremental > >Not if you let amanda do it without interference. > >>> Each fs backed up MUST be able to fit on a tape. You need to >>> specify multiple tapes / use of a tape changer would be >>> desireable. >> >>Ok let me get this straight fs here means filesystem ok? > >yes, and to further define it, each "partition" is a filesystem. And to further clarify, if you use tar, then a subdir of a partition can be a 'filesystem'. I do that here with a 30gb /usr. >> a >> filesystem is per/machine? or is it per time amanda is running? >> So when amanda is running and doing a full backup all the >> filesystems listed in disklist should be able to fit on one >> tape? > >No, each entry's size is limited to one tape's size. With a >changer, amanda can use more than one tape if she needs to because >the current one filled up. In that event, the failed entry in the >disklist will be restarted from scratch on the next tape. > >> Im using the newest beta which will backup to our NAS box with >> 400GB available so i guess its just a question to fool amanda to >> think that im running some big DLT that can have +80GB on one >> tape or so? Is that right? > >No, don't lie to amand about what the tape can hold. In fact, > turn off any compression the drive might have and leave it off > forever, and run the tape-src/tapetype utility after you build > it. It takes a while to run, but then amanda has a very good > idea of the tapes capacity. Enter that output data as the > tapetype spec in your amanda.conf if it doesn't already have one > for your tape. If amanda knows what the tape can hold, the > schedule will probably be adjusted to prevent overflow if it can > be done and stay within that 7 day dumpcycle. > >Amanda will use your favorite compressor if you use the right >'dumptype'. I play mix and match here as some partitions don't >compress well, see the report to see which ones don't. But with > it enabled for about half my system I put 6+ gigs of data on one > DDS-2 tape last night, and had about 1.5 gigs left on that tape. > >With a changer at her disposal, amanda can and will use more than >one tape per run if need be. Enabling that will cause your >magazine reload schedule to be a check it daily operation to make >sure there is enough tapes left for tonight though. -- Cheers, Gene AMD K6-III@500mhz 320M Athlon1600XP@1400mhz 512M 99.14% setiathome rank, not too shabby for a WV hillbilly
Re: special backupcase
On Thursday 05 September 2002 09:29, Thomas Kirk wrote: >Hey Jim > >On Thu, Sep 05, 2002 at 08:01:52AM -0500, Jim Summers wrote: > >First of all thanks for the quick answer! :) > >> I think you will have to set up multiple disklist / configs to >> try and force Amanda to do this. > >Hmm this seems abit complicated to me as im still trying to grasp > the basic concepts of amanda? If you could point me towards some > reading or howto i would be very grateful! > >> Amanda herself determines the level and when that level will be >> done within the dumpcycle. She tries to balance the amount of >> data backed up each run so as to maximize tape usage. Actually, its to equalize tape usage. :-) >So what you are telling me is that i can't control wich days to do > the full backup and which days to do incremental Not if you let amanda do it without interference. >> Each fs backed up MUST be able to fit on a tape. You need to >> specify multiple tapes / use of a tape changer would be >> desireable. > >Ok let me get this straight fs here means filesystem ok? yes, and to further define it, each "partition" is a filesystem. > a > filesystem is per/machine? or is it per time amanda is running? > So when amanda is running and doing a full backup all the > filesystems listed in disklist should be able to fit on one tape? No, each entry's size is limited to one tape's size. With a changer, amanda can use more than one tape if she needs to because the current one filled up. In that event, the failed entry in the disklist will be restarted from scratch on the next tape. > Im using the newest beta which will backup to our NAS box with > 400GB available so i guess its just a question to fool amanda to > think that im running some big DLT that can have +80GB on one > tape or so? Is that right? No, don't lie to amand about what the tape can hold. In fact, turn off any compression the drive might have and leave it off forever, and run the tape-src/tapetype utility after you build it. It takes a while to run, but then amanda has a very good idea of the tapes capacity. Enter that output data as the tapetype spec in your amanda.conf if it doesn't already have one for your tape. If amanda knows what the tape can hold, the schedule will probably be adjusted to prevent overflow if it can be done and stay within that 7 day dumpcycle. Amanda will use your favorite compressor if you use the right 'dumptype'. I play mix and match here as some partitions don't compress well, see the report to see which ones don't. But with it enabled for about half my system I put 6+ gigs of data on one DDS-2 tape last night, and had about 1.5 gigs left on that tape. With a changer at her disposal, amanda can and will use more than one tape per run if need be. Enabling that will cause your magazine reload schedule to be a check it daily operation to make sure there is enough tapes left for tonight though. -- Cheers, Gene AMD K6-III@500mhz 320M Athlon1600XP@1400mhz 512M 99.14% setiathome rank, not too shabby for a WV hillbilly
RE: special backupcase
> > I think you will have to set up multiple disklist / configs > to try and > > force Amanda to do this. > > Hmm this seems abit complicated to me as im still trying to grasp the > basic concepts of amanda? If you could point me towards some reading > or howto i would be very grateful! If your limiting factor is bandwidth, you might be better off fiddling with the tape size until AMANDA is only tying up the DSL line for the amount of time you have the DSL line to yourself. AMANDA automatically balances the full backups across all of the tapes in a tapecycle so that backups are predictable in terms of time and tape used. You should also look at client-side compression, which could save a lot of time depending on how compressible your data is. > > > Amanda herself determines the level and when that level will be done > > within the dumpcycle. She tries to balance the amount of > data backed up > > each run so as to maximize tape usage. > > So what you are telling me is that i can't control wich days > to do the full > backup and which days to do incremental Yes, and most people don't want to, once they get the hang of it. Spreading the level 0 backups across tapes provides you with some protection in case a tape fails, better use of tape capacity, predictable backup durations, and warnings, rather than failures, when your backups get larger. (If you see a lot of backups being bumped, it's might be because your level 0's are getting bigger.) > > > Each fs backed up MUST be able to fit on a tape. You need > to specify > > multiple tapes / use of a tape changer would be desireable. > > Ok let me get this straight fs here means filesystem ok? a filesystem > is per/machine? or is it per time amanda is running? So when amanda is > running and doing a full backup all the filesystems listed in disklist > should be able to fit on one tape? Im using the newest beta which will > backup to our NAS box with 400GB available so i guess its just a > question to fool amanda to think that im running some big DLT that can > have +80GB on one tape or so? Is that right? Each partition on each machine has to be less than the size of one tape. Since you're using a file system for a tape drive, you can set the tape size arbitrarily. Tape size/speed is usually the bottleneck in a backup system, but it sounds like the network is the limit in your case. Use the tape size to control how much is backed up each night. You might also need to set the netusage parameter to some fraction of your DSL speed to keep AMANDA from flooding the line and timing out. Here's how I'd calculate it, your numbers will probably be different: SDSL @ 768 kilobits per second is about 76 kilobytes per second I'd set netusage to 60 KB, or about 80%. (I have confidence in my DSL provider, can you tell?) Assuming you can run your backups for 12 hours per night: 12 * 60 * 60 = 43200 seconds of run time per night 43200 * 60 = 2592000 kilobytes per night, or about 2.5 Gb. > > Cosmic ray particles crashed through the hard disk platter > I prefer alpha particles. They can't get through aluminium foil, but lusers don't know that.
Re: special backupcase
On Thursday 05 September 2002 08:34, Thomas Kirk wrote: >Hey there listmembers > >Im kind of newbie within the backup area so maybe this isnt so > special after all? > >I wanna do a full backup every weekend and incremental rest of the >week which lead me to have > >dumpcycle 7 > >now the trouble is this will make amanda do full backups on sunday >right? The problem is that we are doing our backup over a dslline >which we can't transfer a full backup over in just one day? Whats > the solution for that? What i would like is begin the full backup > on friday afternoon (when everybody is off for weekend) and > continue over the weekend makeing full backup. Start monday with > incremental backup and rotate again the next friday? > >Another thing is a full backup in this case is more than a > tapefull whats the solution for that? That "fixed schedule" s not the design target for amanda. Amanda wants to be in charge of the scheduleing. And she does it in a manner that, when you set the dumpcycle to be 7 days, will guarantee that each entry in the disklist will be covered by a full backup at some point in those 7 days. The idea is to try and equalize the amount of data put on tape each night. This also gets around the full is too big for the media problem very nicely. I'm backing up over 40 gigs of stuff here, on 120 meter DDS-2 media, and the individual nights tape averages maybe 55% used. The general solution for someone who wants a weekly offsite backup would be to go ahead and set the dumpcycle to 7 days, but have 14 or 21 tapes (plus a couple for spares in case a tape gets damaged) in the tapecycle so that last weeks can be brought in for re-use in the following 7 days and this weeks 7 day collection can be taken home, always on the same day of the week, thereby guaranteeing that you have a full backup offsite. With 21 tapes, leave 14 of them offsite except when doing the exchanges, bring in the oldest set of 7 for reuse. It also means that if one tape is foobar, you still have most of your data because its spread over several tapes with no one tape being the key to a full backup image. Left to her own devices, amanda will do a very good job of safeguarding your data, we think its the best in the business, but she does like to do it her way. Amanda cannot span a partition image across more than one tape, but if the media should fill, she will restart from zero that failed dump on the next tape if given permission to use more than one tape per run, see the runtapes option in amanda.conf. -- Cheers, Gene AMD K6-III@500mhz 320M Athlon1600XP@1400mhz 512M 99.14% setiathome rank, not too shabby for a WV hillbilly
Re: special backupcase
On Thu, 2002-09-05 at 08:29, Thomas Kirk wrote: > Hey Jim > > On Thu, Sep 05, 2002 at 08:01:52AM -0500, Jim Summers wrote: > > First of all thanks for the quick answer! :) > > > I think you will have to set up multiple disklist / configs to try and > > force Amanda to do this. > > Hmm this seems abit complicated to me as im still trying to grasp the > basic concepts of amanda? If you could point me towards some reading > or howto i would be very grateful! Not sure on precise locations of readings. I seem to remember some emails going back and forth on this list about doing this type of backup strategy. Maybe there is an archive of this list somewhere. In a nutshell, I think it works with cron to use the correct amdump command to determine which disklist / config files to use and when. > > > Amanda herself determines the level and when that level will be done > > within the dumpcycle. She tries to balance the amount of data backed up > > each run so as to maximize tape usage. > > So what you are telling me is that i can't control wich days to do the full > backup and which days to do incremental > > > Each fs backed up MUST be able to fit on a tape. You need to specify > > multiple tapes / use of a tape changer would be desireable. > > Ok let me get this straight fs here means filesystem ok? a filesystem > is per/machine? or is it per time amanda is running? So when amanda is > running and doing a full backup all the filesystems listed in disklist > should be able to fit on one tape? Im using the newest beta which will > backup to our NAS box with 400GB available so i guess its just a > question to fool amanda to think that im running some big DLT that can > have +80GB on one tape or so? Is that right? Pretty much right. I think amanda defines a filesystem to be a disklist entry. So if you use gtar then the disklist entry can be a sub-directory and that structure(tarball) itself must fit on to one tape. I think if you use dump then the disklist entry must be a mountpoint. I may have misunderstood your set up and if you are going to use a hard disk as your backup device then I can't help since I have zero experience with that. Maybe others on the list can help. > > -- > Venlig hilsen/Kind regards > Thomas Kirk > ARKENA > thomas(at)arkena(dot)com > Http://www.arkena.com > > > BOFH excuse #51: > > Cosmic ray particles crashed through the hard disk platter >
Re: special backupcase
Hey Jim On Thu, Sep 05, 2002 at 08:01:52AM -0500, Jim Summers wrote: First of all thanks for the quick answer! :) > I think you will have to set up multiple disklist / configs to try and > force Amanda to do this. Hmm this seems abit complicated to me as im still trying to grasp the basic concepts of amanda? If you could point me towards some reading or howto i would be very grateful! > Amanda herself determines the level and when that level will be done > within the dumpcycle. She tries to balance the amount of data backed up > each run so as to maximize tape usage. So what you are telling me is that i can't control wich days to do the full backup and which days to do incremental > Each fs backed up MUST be able to fit on a tape. You need to specify > multiple tapes / use of a tape changer would be desireable. Ok let me get this straight fs here means filesystem ok? a filesystem is per/machine? or is it per time amanda is running? So when amanda is running and doing a full backup all the filesystems listed in disklist should be able to fit on one tape? Im using the newest beta which will backup to our NAS box with 400GB available so i guess its just a question to fool amanda to think that im running some big DLT that can have +80GB on one tape or so? Is that right? -- Venlig hilsen/Kind regards Thomas Kirk ARKENA thomas(at)arkena(dot)com Http://www.arkena.com BOFH excuse #51: Cosmic ray particles crashed through the hard disk platter
Re: special backupcase
On Thu, 2002-09-05 at 07:34, Thomas Kirk wrote: > Hey there listmembers > > Im kind of newbie within the backup area so maybe this isnt so special > after all? > > I wanna do a full backup every weekend and incremental rest of the > week which lead me to have I think you will have to set up multiple disklist / configs to try and force Amanda to do this. > > dumpcycle 7 > > now the trouble is this will make amanda do full backups on sunday > right? The problem is that we are doing our backup over a dslline Amanda herself determines the level and when that level will be done within the dumpcycle. She tries to balance the amount of data backed up each run so as to maximize tape usage. > which we can't transfer a full backup over in just one day? Whats the > solution for that? What i would like is begin the full backup on > friday afternoon (when everybody is off for weekend) and continue over > the weekend makeing full backup. Start monday with incremental backup > and rotate again the next friday? > > Another thing is a full backup in this case is more than a tapefull > whats the solution for that? Each fs backed up MUST be able to fit on a tape. You need to specify multiple tapes / use of a tape changer would be desireable. > > -- > Venlig hilsen/Kind regards > Thomas Kirk > ARKENA > thomas(at)arkena(dot)com > Http://www.arkena.com > > > BOFH excuse #362: > > Plasma conduit breach >
