Re: Backup plan and big filesystems
On Wed, Aug 23, 2006 at 09:15:30AM -0400, Joshua Baker-LePain wrote: > On Wed, 23 Aug 2006 at 6:23am, Joshua Baker-LePain wrote > > >On Wed, 23 Aug 2006 at 1:53am, Jon LaBadie wrote > > > >>Others have a cost consideration for how many tapes > >>physically are offsited (can offsite be a verb? :) > > > >Verbing weirds language. ;) > > And lest folks thing I'm trying to take credit for that statement, I'll > state that the attribution is the great philosopher Calvin. No, not that > one, the other one. > Wish I read that comic strip regularly, great wisdom there! -- Jon H. LaBadie [EMAIL PROTECTED] JG Computing 4455 Province Line Road(609) 252-0159 Princeton, NJ 08540-4322 (609) 683-7220 (fax)
Re: Backup plan and big filesystems
On Wed, 23 Aug 2006 at 6:23am, Joshua Baker-LePain wrote On Wed, 23 Aug 2006 at 1:53am, Jon LaBadie wrote Others have a cost consideration for how many tapes physically are offsited (can offsite be a verb? :) Verbing weirds language. ;) And lest folks thing I'm trying to take credit for that statement, I'll state that the attribution is the great philosopher Calvin. No, not that one, the other one. *sigh* Never post before your morning coffee... -- Joshua Baker-LePain Department of Biomedical Engineering Duke University
Re: Backup plan and big filesystems
On Wed, 23 Aug 2006 at 1:53am, Jon LaBadie wrote Others have a cost consideration for how many tapes physically are offsited (can offsite be a verb? :) Verbing weirds language. ;) -- Joshua Baker-LePain Department of Biomedical Engineering Duke University
Re: Backup plan and big filesystems
On Wed, Aug 23, 2006 at 09:43:14AM +0700, Olivier Nicole wrote: > Hi, > > Time to add my 2 cents to that thread, and ask for advice from Amanda > gurus too. > > The way I solvedthe offsite tapes thing is the following: > > I declared: > > dumpcycle 1 week > runspercycle 5 > tapecycle 6 tapes > > I labeled 6*3 tapes, with labels like set-1-00 to set-1-05, set-2-00 > to set-2-05 and set-3-00 to set-3-05. Only set-1 is marked for reuse, > when I reach tape set-1-05, I mark all set-1 as no-reuse and set-2 as > reuse. And etc. > > This way I should have 2 complete cycle, including full and incr to be > offsited. > > Am I doing anything wrong? I find this much straight forward than > having 2 configuration, one for full and one for incr. > Nothing wrong. Particularly if it works for you. Some sites want only to offsite a set of full dumps. A second config works well for this. Others have a cost consideration for how many tapes physically are offsited (can offsite be a verb? :) I recall one poster saying the commercial storage facility charged by the tape stored and extra for each tape moved back and forth. As you are rotating 18 tapes (3 sets of 6 each) and have 4 runs per cycle, you really have a total of 3.6 "dump cycles" worth of tape, not 3. I'd worry about my schedule for no-reuse/reuse marking. It is not regular. It would be after Monday (Tuesday) to start set 2, then after Wednesday for set 3. Set 1 starts again on a Friday. Opps, that is after Friday, so on Monday. What if you forget to do the remarking of the tapelist? It will go back to using tape 0 of the same set. Perhaps have your cronjob automatically mark the tape used that day "noreuse" and mark the 6th oldest one "reuse". For you, what is the value of having 12 of your 18 tapes marked "noreuse"? Is it necessary to keep setting and resetting them? When I backed up to DDS3 tapes, my changer had magazines that held 6 tapes. So I configured 6 runs/week, Saturday night was skipped and Sunday night (actually Monday morning) collected the 2 day weekend changes. My "SOHO" network was so valuable that of course I had to offsite tapes -- to my neighbors home :) I changed magazines anytime over the weekend. I kept 4 magazines in rotation. Onsite were the one in use, the one to be used next (oldest dumps), and the most recently used for recovery of recent stuff. Offsite was the one from two weeks before. Eventually my magazines no longer synchronized with the weekends and I gave up the offsiting. I'd change magazines when the report said "dumps left in holding disk". Or maybe a day or two later as I had enought holding disk for over a week. I think I still have some archive dumps (different config and magazines) at my neighbors house. -- Jon H. LaBadie [EMAIL PROTECTED] JG Computing 4455 Province Line Road(609) 252-0159 Princeton, NJ 08540-4322 (609) 683-7220 (fax)
Re: Backup plan and big filesystems
Hi, Time to add my 2 cents to that thread, and ask for advice from Amanda gurus too. The way I solvedthe offsite tapes thing is the following: I declared: dumpcycle 1 week runspercycle 5 tapecycle 6 tapes I labeled 6*3 tapes, with labels like set-1-00 to set-1-05, set-2-00 to set-2-05 and set-3-00 to set-3-05. Only set-1 is marked for reuse, when I reach tape set-1-05, I mark all set-1 as no-reuse and set-2 as reuse. And etc. This way I should have 2 complete cycle, including full and incr to be offsited. Am I doing anything wrong? I find this much straight forward than having 2 configuration, one for full and one for incr. Best regards, Olivier
Re: Backup plan and big filesystems
Fabio Corazza <[EMAIL PROTECTED]> a écrit sur 22/08/2006 12:26:36 : > Cyrille Bollu wrote: > >> But I still have one question: if the collection of the tapes is on > >> Sunday (or any other planned day of the week), how can I make sure that > >> Amanda will do backups within that day? > > > > What do you mean? I don't understand? > > > > Amanda is launched via a cron job so you decide when you want to backup. > > I though that it was deciding by itself when to backup data since I've > heard about "dynamic scheduling"... using cron jobs doesn't mean it's > dynamic... am I missing something? Amanda decides *what* data are being backuped at each run (ie: this DLE is full-backup, this other one is incremental level 1,...). But the runs themself are scheduled via cron. But that's standard Amanda configuration. In your case, you don't need to care since you will be doing full backups only (on sunday). > > > Yes, adding a tape to a set is as simple as "amlabel > > ". Amanda will know that a new active tape has been > > added and it will use it when needed (I think directly at the next run). > > Ok, and can I also modify the runtapes (in case my full dump of > everything would require more than one tape) as well as tapcycle and it > will be used in the next run? Yes, probably, one has to adjust tapecycle when adding a tape with amlabel. Also, I don't think modifying runtapes would break Amanda... Cyrille
Re: Backup plan and big filesystems
> > But I still have one question: if the collection of the tapes is on > Sunday (or any other planned day of the week), how can I make sure that > Amanda will do backups within that day? What do you mean? I don't understand? Amanda is launched via a cron job so you decide when you want to backup. > > Also, seems like for the moment we don't need any off-site and the > amount of data in the filesystem will not grow above 400GB within 3 or 4 > months. > > The question is: can I configure for the moment Amanda, to have just one > set that always stays in the robot, with 7x400GB cartridges for the > daily + 1 for the full backup + 1? Can I expand in a later moment the > configuration with more tapes and an additional configuration for the > off-site backup? This is because our vendor sells cartridges in pack of > 5 and for the moment would be enough just 10 cartridges, then later we > can expand the storage space with more cartridges if needed, but this > will not happen within the next 3-4 months like I said. > > Yes, adding a tape to a set is as simple as "amlabel ". Amanda will know that a new active tape has been added and it will use it when needed (I think directly at the next run). Cyrille
Re: Backup plan and big filesystems
Cyrille Bollu wrote: >> > What they definitely need to know is which tapes are to be pulled off. >> > All the tapes in the changer will have a bar code label. Can I instruct >> > Amanda to send an email with which tapes have been used for a determined >> > backup (the full weekly off-site, in this case) so that the people there >> > will have an idea of which tapes need to be substituted? >> >> How about two Amanda configs "daily" and "archive", with "labelstr" >> different for each of them (reg.exp. like: "Newbay-Daily[0-9]*" and >> "[Newbay-Archive[0-9]*"). And then just write that label on the sticker >> too, readable by a human. >> >> Then just the tapes labeled "*Archive*" need to be pulled out... >> > > That's what I'm trying to say in my bad English ;-) Thanks everybody for the help (Cyrille your English seems great to me :-), I think that this is the best solution. Having two different sets, with different tapes (labeled both human-readably and with amlabel), with one that always stays inside the robot, and the other to be exported off-site and rotated. But I still have one question: if the collection of the tapes is on Sunday (or any other planned day of the week), how can I make sure that Amanda will do backups within that day? Also, seems like for the moment we don't need any off-site and the amount of data in the filesystem will not grow above 400GB within 3 or 4 months. The question is: can I configure for the moment Amanda, to have just one set that always stays in the robot, with 7x400GB cartridges for the daily + 1 for the full backup + 1? Can I expand in a later moment the configuration with more tapes and an additional configuration for the off-site backup? This is because our vendor sells cartridges in pack of 5 and for the moment would be enough just 10 cartridges, then later we can expand the storage space with more cartridges if needed, but this will not happen within the next 3-4 months like I said. Regards, -- Fabio Corazza - Engineering NewBay Software, Ltd. Wilson House, Fenian Street, Dublin 2, Ireland Phone: +353 1 634 5490 - e-mail: [EMAIL PROTECTED]
Re: Backup plan and big filesystems
> > > > What they definitely need to know is which tapes are to be pulled off. > > All the tapes in the changer will have a bar code label. Can I instruct > > Amanda to send an email with which tapes have been used for a determined > > backup (the full weekly off-site, in this case) so that the people there > > will have an idea of which tapes need to be substituted? > > How about two Amanda configs "daily" and "archive", with "labelstr" > different for each of them (reg.exp. like: "Newbay-Daily[0-9]*" and > "[Newbay-Archive[0-9]*"). And then just write that label on the sticker > too, readable by a human. > > Then just the tapes labeled "*Archive*" need to be pulled out... > That's what I'm trying to say in my bad English ;-)
Re: Backup plan and big filesystems
On 2006-08-21 16:45, Fabio Corazza wrote: I'd rather go with 3 chunks each of 400GB (at least, the first two of 400GB and the third with the remaining). Or if this does not fit well for Amanda, I can create multiple chunks of 200GB, but not less, since this is the size of the virtual disks on the SAN, that I striped with the volume manager. Please bear with my curiosity (:-)): if I strictly need a BIG filesystem to be backed up, will Amanda behave _so_ badly? It's not so much Amanda. THe backup will probably work fine. But (just experienced this myself in a little smaller scale :-) !!) the restore of such data will take ages too. For some small remote office I make a backup to vtapes on an external USB-disk. Because the server is rather old too, it has an 1.1 usb only, capable of transferring 1 Mbyte/sec. The largest DLE is currently 65 Gbyte, compressed down to 40 GByte. According to Murphy's laws, the files to restore are usually located at the end of that image, so reading such a gnutar image off the disk takes about 4 seconds, or a little bit over 11 hours :-) "But I only need those 20 files, they are hardly 1 Mbyte each" Luckily the files were restored after about 1/3th of the image, but still taking a few hours! I will soon split that DLE up into many smaller chunks! Now, what it mainly concerns me, are the off-site tapes. Like you said probably the best way to go would be having two different tape sets: 1 for incremental+fullweekly, and 1 for fullweekly to be taken off-site. While the first set (incremental+full) will be entirely managed by Amanda, I'd like to schedule the off-site backup (off-site meant as the tapes have to be taken off-site, please don't confuse) to be lasted not over a specific day of the week. In that day some guys in the data center will be pulling off the off-site tapes and inserting new ones. The next week they will be reinserting the previous tapes, etc. Basically we have two tape sets to be rotated weekly. What they definitely need to know is which tapes are to be pulled off. All the tapes in the changer will have a bar code label. Can I instruct Amanda to send an email with which tapes have been used for a determined backup (the full weekly off-site, in this case) so that the people there will have an idea of which tapes need to be substituted? How about two Amanda configs "daily" and "archive", with "labelstr" different for each of them (reg.exp. like: "Newbay-Daily[0-9]*" and "[Newbay-Archive[0-9]*"). And then just write that label on the sticker too, readable by a human. Then just the tapes labeled "*Archive*" need to be pulled out... -- Paul Bijnens, xplanation Technology ServicesTel +32 16 397.511 Technologielaan 21 bus 2, B-3001 Leuven, BELGIUMFax +32 16 397.512 http://www.xplanation.com/ email: [EMAIL PROTECTED] *** * I think I've got the hang of it now: exit, ^D, ^C, ^\, ^Z, ^Q, ^^, * * F6, quit, ZZ, :q, :q!, M-Z, ^X^C, logoff, logout, close, bye, /bye, * * stop, end, F3, ~., ^]c, +++ ATH, disconnect, halt, abort, hangup, * * PF4, F20, ^X^X, :D::D, KJOB, F14-f-e, F8-e, kill -1 $$, shutdown, * * init 0, kill -9 1, Alt-F4, Ctrl-Alt-Del, AltGr-NumLock, Stop-A, ... * * ... "Are you sure?" ... YES ... Phew ... I'm out * ***
Re: Backup plan and big filesystems
On 2006-08-21 17:07, Cyrille Bollu wrote:
[EMAIL PROTECTED] a écrit sur 21/08/2006 16:45:18 :
[ ??? owner-amanda-users ??? I believe the quoting is wrong ]
>
> But if I stick with just one big GFS fs, I think I will have just one
> DLE and then Amanda will take care to span the tar volume into multiple
> tapes, when it receives an end-of-media message. Please correct me :-)
Yes, but AFAIK that's a rather new feature that doesn't look easy to set
up (that's my stupid newbie opinion :-).
define dumptype comp-tar-split {
comp-tar
tape_splitsize 1g
split_diskbuffer "/space/amandahold"
fallback_splitsize 256m
}
And voila... That's enough.
--
Paul Bijnens, xplanation Technology ServicesTel +32 16 397.511
Technologielaan 21 bus 2, B-3001 Leuven, BELGIUMFax +32 16 397.512
http://www.xplanation.com/ email: [EMAIL PROTECTED]
***
* I think I've got the hang of it now: exit, ^D, ^C, ^\, ^Z, ^Q, ^^, *
* F6, quit, ZZ, :q, :q!, M-Z, ^X^C, logoff, logout, close, bye, /bye, *
* stop, end, F3, ~., ^]c, +++ ATH, disconnect, halt, abort, hangup, *
* PF4, F20, ^X^X, :D::D, KJOB, F14-f-e, F8-e, kill -1 $$, shutdown, *
* init 0, kill -9 1, Alt-F4, Ctrl-Alt-Del, AltGr-NumLock, Stop-A, ... *
* ... "Are you sure?" ... YES ... Phew ... I'm out *
***
Re: Backup plan and big filesystems
[EMAIL PROTECTED] a écrit sur 21/08/2006 16:45:18 : > Jon LaBadie wrote: > [snip] > > If go with amanda you definitely should do that GFS filesystem as > > multiple DLEs. If I got your data size correct, you have about > > 1 TB of GFS data plus 0.4 TB of misc OS data, about 1.4TB total. > > This in the case I will decide to split the actual filesystem into more > than one, that means I will need to create multiple DLEs. > > But if I stick with just one big GFS fs, I think I will have just one > DLE and then Amanda will take care to span the tar volume into multiple > tapes, when it receives an end-of-media message. Please correct me :-) Yes, but AFAIK that's a rather new feature that doesn't look easy to set up (that's my stupid newbie opinion :-). > (snip: don't understand) > > Now, what it mainly concerns me, are the off-site tapes. > > Like you said probably the best way to go would be having two different > tape sets: 1 for incremental+fullweekly, and 1 for fullweekly to be > taken off-site. It's not exactly what I said. I said: 1 set for incremental only + 1 set for full only. You never export the incremental set; it stays forever in your robot. Your full-only set is made of 2 times the number of tapes needed for a full dump. Every week, Amanda uses half of your "full tapes" to do its full backups and every week you replace these tapes by the other one from the "full set". This way you have a weekly full off-site. And Amanda reports tell you exactly what tape it used and what tape it expect. example of Amanda report (from my config): These dumps were to tape daily-backup05. The next tape Amanda expects to use is: daily-backup06. STATISTICS: Total Full Daily Estimate Time (hrs:min) 0:16 Run Time (hrs:min) 1:42 Dump Time (hrs:min) 1:26 1:18 0:08 Output Size (meg) 99549.4 93266.7 6282.7 Original Size (meg) 99549.4 93266.7 6282.7 Avg Compressed Size (%) -- -- -- (level:#disks ...) Filesystems Dumped 23 15 8 (1:8) Avg Dump Rate (k/s) 19865.0 20498.5 13617.2 (ndlr: wow so slw!) Tape Time (hrs:min) 1:26 1:18 0:08 Tape Size (meg) 99549.4 93266.7 6282.7 Tape Used (%) 22.1 20.7 1.4 (level:#disks ...) Filesystems Taped 23 15 8 (1:8) Avg Tp Write Rate (k/s) 19838.7 20469.8 13609.9 USAGE BY TAPE: Label Time Size % Nb daily-backup05 1:26 99549.4 22.1 23 The downside is that the "incremental only" tapes are under-used. However, John estimated that your daily dumps would be about 300GB large if your DLE were correctly balanced. In that case, having a weekly full off-site would only require 2*7=14 LTO3 tapes. Isn't that good enough?
Re: Backup plan and big filesystems
Jon LaBadie wrote: [snip] > If go with amanda you definitely should do that GFS filesystem as > multiple DLEs. If I got your data size correct, you have about > 1 TB of GFS data plus 0.4 TB of misc OS data, about 1.4TB total. This in the case I will decide to split the actual filesystem into more than one, that means I will need to create multiple DLEs. But if I stick with just one big GFS fs, I think I will have just one DLE and then Amanda will take care to span the tar volume into multiple tapes, when it receives an end-of-media message. Please correct me :-) > With amanda's dynamic scheduling, and say daily runs with a dump > cycle of 1 week (7 runs/week, full dump of everything each week), > a perfect balance would have 1.4TB/7 or 200GB/day of full dumps. > Plus some incremental data plus it is likely that some DLE will > get more than one full dump during the week. But it sounds like > things would fit onto a single LTO-3 tape, even without considering > compression. Your daily dumps would probably be about 300GB. As far of my knowledge this should be: dumpcycle 1 week runspercycle 7 tapecycle 10 tapes 10 tapes are for the 7 tapes rotated weekly plus 3 for the size of the GFS filesystem and possible errors. Note that the incremental backups of the local servers will be probably very small (we are talking about a few MBs for each server - those are mostly log files). Due to the nature of the platform the data mainly changes in the SAN (GFS space). > Amanda schedules best when it has a goodly number of DLEs to balance. > It is hard to balance 4 DLEs of 1TB, 100GB, 100Gb, 100GB. :) > You probaly have about 15-20 file systems among your 4 or 5 clients > (will the server also be a client) and if you setup 8-10 DLEs from > your 1TB GFS system, you should be golden. I'd rather go with 3 chunks each of 400GB (at least, the first two of 400GB and the third with the remaining). Or if this does not fit well for Amanda, I can create multiple chunks of 200GB, but not less, since this is the size of the virtual disks on the SAN, that I striped with the volume manager. Please bear with my curiosity (:-)): if I strictly need a BIG filesystem to be backed up, will Amanda behave _so_ badly? Now, what it mainly concerns me, are the off-site tapes. Like you said probably the best way to go would be having two different tape sets: 1 for incremental+fullweekly, and 1 for fullweekly to be taken off-site. While the first set (incremental+full) will be entirely managed by Amanda, I'd like to schedule the off-site backup (off-site meant as the tapes have to be taken off-site, please don't confuse) to be lasted not over a specific day of the week. In that day some guys in the data center will be pulling off the off-site tapes and inserting new ones. The next week they will be reinserting the previous tapes, etc. Basically we have two tape sets to be rotated weekly. What they definitely need to know is which tapes are to be pulled off. All the tapes in the changer will have a bar code label. Can I instruct Amanda to send an email with which tapes have been used for a determined backup (the full weekly off-site, in this case) so that the people there will have an idea of which tapes need to be substituted? Last but not least.. I have a total amount of available cartridges of 25 LTO-3 Ultrium (400GB). As far as I know, approx 10 will be needed for the "in-site" backup, while for the full I will need approx.. 3-4 tapes * 2 = 8? Then a total amount of 18 tapes should be sufficient. 14 to be put on the changer and 4 for the full backup to be rotated during the week by the people in the data center. > Prescription for stress: > 5 amanda installations, bed rest, and call us in the morning. I'd love to have a doctor like you.. but you'd hate to have a patient like me.. ;-) -- Fabio Corazza - Engineering NewBay Software, Ltd. Wilson House, Fenian Street, Dublin 2, Ireland Phone: +353 1 634 5490 - e-mail: [EMAIL PROTECTED]
Re: Backup plan and big filesystems
On Fri, Aug 18, 2006 at 02:06:48PM +0100, Fabio Corazza wrote: > Sorry for the length of the email, but it's been a bit difficult to > explain everything... :-) > Followed by a long-winded answer without apologies. > > Hi there, > I'm wondering which is the best backup technique that I can implement > on a Linux-only servers production environment. > > I was wondering initially, also under suggestion of some colleagues, to > stick with a cron-scripted solution, since they had the impression that > Amanda would just over complicate everything. Once installed, configured and running, amanda needs little admin. Mostly changing tapes and monitoring the daily emails. You can find evidence on this list of people who have had almost zero trouble setting up their initial amanda installation and others who have had considerable teething pains. > > Anyway this would require initial engineering time, like writing cron > scripts that would manage dump and tar utilities together mtx to manage > all the stuff. At this point in time, I'd rather go with a ready-to-go > solution like Amanda, even if initially we still would need some time to > spend on the configuration. Also, this will give us the fancy of > detailed email reports, that are always a good thing. Yes, the reports are a bonus. Also consider the detailed activity logs and debugging files with more info than you really want, but which are there when you do need them. And indexes of your tapes and archive contents. Would your scripts allow you to browse your backups for a specific file(s) to recover, or would you have to manually look for them and hope you get the right tape and tape file and command line syntax and typo-less command line entry? Could you do recovery from the backup clients as well as from the amanda server? Will your scripts make sure you don't overwrite a tape before it is appropriate to do so? Will your cron scripts automatically handle the occasional hiccup? Like a host or network being down at the time of backup or tape problems? Will they automatically adjust incremental levels based on the amount of data change? There are benefits, but ... > > This is what we have to backup: 4 (four) production servers (2x > application & 2x database) full filesystems dump (/, /var, etc), and > some GFS filesystem stored on an iSCSI SAN, mounted as read-only on the > admin server, where Amanda server will be running. > > We will use tape as backup medium, with a Dell PowerVault 124T attached > to the SCSI controller of the admin server. This tape library will be > capable of handling a total size of 6.4TB native uncompressed data with > 2 magazines of 8 slots each using LTO-3 Ultrium 400GB cartridges. > ... > Other than this, I'm trying to figure out which is the best backup plan. > I'd go with a weekly full backup for everything (servers local > filesystems & GFS volumes) and an incremental daily. We will use 2 > different tape sets, to export physically tape cartridges from the data > center every week for data safety. > > What I'm a bit confused about is how to manage the tape cycles during > the week, and how to handle backup of big size filesystems (in our case > GFS). Actually the filesystem is of around 1TB, but I'm guessing if it > would be better to split it in different mountpoints, each of 400GB. > This would probably give some benefits for the backup, since this is the > exact size of our cartridges. > > If I'm not wrong, this is what it should be done: > [ a traditional backup scheme description snipped ] While you can do traditional backup scheduling (monster full dumps one day and incremental the others) amanda is best used the way it was designed, by allowing amanda to dynamically scheduled your dumps and spread the full and incrementals over the entire week. Using any traditional scheme will always have you struggleing with amanda to "do it your way" rather than amanda's. It can be done, and you get the other nice benefits, but it isn't pretty. The backup unit of amanda is called a disklist entry (DLE) based on the configuration file name. A DLE may be an entire filesystem or one or more directory trees. You provide amanda a framework from which to do its scheduling -- how frequently will you be doing dumps, how regularly do you need a full dump, up to how many tapes can it use for a single dump, how many tapes total are in the rotation. Things like this and many many others can be specified globally and for many of the configuration parameters can also be unique to each DLE (if you want, some may get full dumps monthly, others weekly, others daily, some only full dumps, skip the incrementals, some only incrementals, full dumps only on demand, some software compression, others not, etc.). Once se
Re: RE Backup plan and big filesystems
On 2006-08-18 16:06, Fabio Corazza wrote: Cyrille Bollu wrote: No. Amanda does not allow appending dumps on a same cartidge. That's by design. Neither can I put all the full dumps into the same cartridge? Since full dumping a server would require around 10/20GB, the cartridge would be almost empty! Read this page why Amanda does it that way: http://wiki.zmanda.com/index.php/Why_does_Amanda_not_append_to_a_tape%3F In the same page is also a "plan B" described if you have indeed invested all your money in an expensive tape drive and you have no budget left to buy cartridges... One more improvement to that "plan B": When using Amanda 2.5.1 or later, you can override some parameters from amanda.conf on the commandline. Using this method you can run with a non-existing tape device in "amanda.conf" instead of not inserting a tape, and once a week run with the real tapedevice specified on the cmd line: "amdump -o tapedev=/dev/nst0" (Note 2.5.1 is currently in beta test.) I think personally that is a waste of space using a single 400GB cartridge for every full dump. I'd rather use the same cartridge for every all the server... ...and losing all of your backups by one simple poweroutage or operator error. Wasn't this a "production" environment? What is the cost of losing all the backups of the production environment? Or having only a backup from one month ago? -- Paul Bijnens, xplanation Technology ServicesTel +32 16 397.511 Technologielaan 21 bus 2, B-3001 Leuven, BELGIUMFax +32 16 397.512 http://www.xplanation.com/ email: [EMAIL PROTECTED] *** * I think I've got the hang of it now: exit, ^D, ^C, ^\, ^Z, ^Q, ^^, * * F6, quit, ZZ, :q, :q!, M-Z, ^X^C, logoff, logout, close, bye, /bye, * * stop, end, F3, ~., ^]c, +++ ATH, disconnect, halt, abort, hangup, * * PF4, F20, ^X^X, :D::D, KJOB, F14-f-e, F8-e, kill -1 $$, shutdown, * * init 0, kill -9 1, Alt-F4, Ctrl-Alt-Del, AltGr-NumLock, Stop-A, ... * * ... "Are you sure?" ... YES ... Phew ... I'm out * ***
Re: RE Backup plan and big filesystems
[EMAIL PROTECTED] a écrit sur 18/08/2006 16:06:45 : > Cyrille Bollu wrote: > > No. Amanda does not allow appending dumps on a same cartidge. That's by > > design. > > Neither can I put all the full dumps into the same cartridge? Since full > dumping a server would require around 10/20GB, the cartridge would be > almost empty! > > > I think it's possible to run an Amanda configuration that only does > > incremental daily backups and another one that run only full weekly > > backups. > > > > Then you would only export the weekly full backups. > > > > You will probably use in that case 2(weeks)*X(tapes_per_full) + > > 7(days)*X(tapes_per_incremental). So let say 9 tapes instead of 15. > > > > What do you think? > > I think personally that is a waste of space using a single 400GB > cartridge for every full dump. I'd rather use the same cartridge for > every all the server... Sorry for my bad english: Amanda cannot append on the same tape dumps from 2 different days but Amanda do append dumps from the same day. > > > On Amanda side, it would probably not be that tricky. Probably the most > > trickiest part of the job will be to handle these 2 sets of tapes > > (weekly and daily) in your 8-tapes robot. > > It has 16 tapes if this can help. It's probably enough. Say you have 2 sets of tapes: incremental1-8 and full1-4 (12 tapes). You put the 8 "incremental" tapes in the robot together with 2 (or wathever is necessary to hold all your data) "full" tapes. You thus use 10 drives in the robot. 1) Every day Amanda will search in the "incremental" set for a so-called "active tape" (a tape that doesn't hold data younger than X (eg 7) days) and write incremental dumps to it. 2) Every week amanda Amanda will search in the "full" set for an active tape to write its full dumps. 3) Every week you change the 2 full tapes (rotation with 4 tapes)
Re: RE Backup plan and big filesystems
Cyrille Bollu wrote: > No. Amanda does not allow appending dumps on a same cartidge. That's by > design. Neither can I put all the full dumps into the same cartridge? Since full dumping a server would require around 10/20GB, the cartridge would be almost empty! > I think it's possible to run an Amanda configuration that only does > incremental daily backups and another one that run only full weekly > backups. > > Then you would only export the weekly full backups. > > You will probably use in that case 2(weeks)*X(tapes_per_full) + > 7(days)*X(tapes_per_incremental). So let say 9 tapes instead of 15. > > What do you think? I think personally that is a waste of space using a single 400GB cartridge for every full dump. I'd rather use the same cartridge for every all the server... > On Amanda side, it would probably not be that tricky. Probably the most > trickiest part of the job will be to handle these 2 sets of tapes > (weekly and daily) in your 8-tapes robot. It has 16 tapes if this can help. -- Fabio Corazza - Engineering NewBay Software, Ltd. Wilson House, Fenian Street, Dublin 2, Ireland Phone: +353 1 634 5490 - e-mail: [EMAIL PROTECTED]
Tr : Re: RE Backup plan and big filesystems
Let's keep the other amanda users informed as I might say stupid things :-) - Réacheminé par Cyrille Bollu/FHQ/USERS/FEDASIL le 18/08/2006 15:51 - Cyrille Bollu/FHQ/USERS/FEDASIL a écrit sur 18/08/2006 15:50:41 : > Fabio Corazza <[EMAIL PROTECTED]> a écrit sur 18/08/2006 15:33:19 : > > > Cyrille Bollu wrote: > > > Forget about weekly full. Amanda uses a completly different logic: > > > > > > Says you have 5 servers to backup and want every server to be full > > > backuped at least once a week. Amanda will probably do something like: > > > > > > full backup of server 1 on monday/ other servers incremental > > > full backup of server 2 on tuesday/ other servers incremental > > > full backup of server 3 on wednesday/ other servers incremental > > > full backup of server 4 on thursday/ other servers incremental > > > full backup of server 5 on friday/ other servers incremental > > > > > > You don't have to tell Amanda what server has to be backuped when. > > > Amanda will do it for you... It will simply respect your requirement of > > > "at least every server must receive a full backup once a week" > > > > Ok, this sounds nice, even if I'm not going to backup an office > > environment but a 24/7 production one. So we need to consider Saturday > > as well as Sunday inside the backup schedule. > > > > > Say you need 1 cartidge per day, with your backup scenario you will need > > > at least 2(weeks)*5(days)*1(tape_per_run)+1 = 11 tapes > > > > What "+1" means? Also, considering 7 weekdays this is going to be pretty > > consuming as of cartridges numbers. > > > Can't I manage a better plan, say, put full dumps and incremental > dumps into the same cartridge > > No. Amanda does not allow appending dumps on a same cartidge. That'sby design. > > > ... > > I think it's possible to run an Amanda configuration that only does > incremental daily backups and another one that run only full weekly backups. > > Then you would only export the weekly full backups. > > You will probably use in that case 2(weeks)*X(tapes_per_full) + > 7(days)*X(tapes_per_incremental). So let say 9 tapes instead of 15. > > What do you think? > > On Amanda side, it would probably not be that tricky. Probably the > most trickiest part of the job will be to handle these 2 sets of > tapes (weekly and daily) in your 8-tapes robot. > > Cyrille
RE Backup plan and big filesystems
[EMAIL PROTECTED] a écrit sur 18/08/2006 15:06:48 : > Sorry for the length of the email, but it's been a bit difficult to > explain everything... :-) > > > Hi there, > I'm wondering which is the best backup technique that I can implement > on a Linux-only servers production environment. > > I was wondering initially, also under suggestion of some colleagues, to > stick with a cron-scripted solution, since they had the impression that > Amanda would just over complicate everything. > > Anyway this would require initial engineering time, like writing cron > scripts that would manage dump and tar utilities together mtx to manage > all the stuff. At this point in time, I'd rather go with a ready-to-go > solution like Amanda, even if initially we still would need some time to > spend on the configuration. Also, this will give us the fancy of > detailed email reports, that are always a good thing. Amanda is easy to set up. Most probably, easier than writing a cron script. > ... > > > Other than this, I'm trying to figure out which is the best backup plan. > I'd go with a weekly full backup for everything (servers local > filesystems & GFS volumes) and an incremental daily. We will use 2 > different tape sets, to export physically tape cartridges from the data > center every week for data safety. Forget about weekly full. Amanda uses a completly different logic: Says you have 5 servers to backup and want every server to be full backuped at least once a week. Amanda will probably do something like: full backup of server 1 on monday/ other servers incremental full backup of server 2 on tuesday/ other servers incremental full backup of server 3 on wednesday/ other servers incremental full backup of server 4 on thursday/ other servers incremental full backup of server 5 on friday/ other servers incremental You don't have to tell Amanda what server has to be backuped when. Amanda will do it for you... It will simply respect your requirement of "at least every server must receive a full backup once a week" > > What I'm a bit confused about is how to manage the tape cycles during > the week, and how to handle backup of big size filesystems (in our case > GFS). Actually the filesystem is of around 1TB, but I'm guessing if it > would be better to split it in different mountpoints, each of 400GB. > This would probably give some benefits for the backup, since this is the > exact size of our cartridges. > > If I'm not wrong, this is what it should be done: > > Monday 00:00 full servers filesystem dump into 1 cartridge; full GFS tar > into multiple tapes OR 1 cartridge for every single 400GB GFS filesystem > > Tue-Sun 00:00 incremental dump of servers filesystem; incremental GFS > tar as above > > The questions are: > > - How many cartridges do I need? Say you need 1 cartidge per day, with your backup scenario you will need at least 2(weeks)*5(days)*1(tape_per_run)+1 = 11 tapes > - How cartridges should be rotated during the week? Let Amanda do its job. > - Is it possible to use a different set of tape for every week, e.g. two > sets to be rotated weekly using a full backup every monday? Don't think this way :-) > > - Is it better to split the GFS filesystem into multiple 400GB volumes > to improve the ease of backup? Probably
Backup plan and big filesystems
Sorry for the length of the email, but it's been a bit difficult to explain everything... :-) Hi there, I'm wondering which is the best backup technique that I can implement on a Linux-only servers production environment. I was wondering initially, also under suggestion of some colleagues, to stick with a cron-scripted solution, since they had the impression that Amanda would just over complicate everything. Anyway this would require initial engineering time, like writing cron scripts that would manage dump and tar utilities together mtx to manage all the stuff. At this point in time, I'd rather go with a ready-to-go solution like Amanda, even if initially we still would need some time to spend on the configuration. Also, this will give us the fancy of detailed email reports, that are always a good thing. This is what we have to backup: 4 (four) production servers (2x application & 2x database) full filesystems dump (/, /var, etc), and some GFS filesystem stored on an iSCSI SAN, mounted as read-only on the admin server, where Amanda server will be running. We will use tape as backup medium, with a Dell PowerVault 124T attached to the SCSI controller of the admin server. This tape library will be capable of handling a total size of 6.4TB native uncompressed data with 2 magazines of 8 slots each using LTO-3 Ultrium 400GB cartridges. While the local filesystems of all the servers would fit on a single 400GB cartridges (leaving most of the space free), I'm wondering how to manager the backup of the GFS volumes. GFS infact doesn't have a "dump" tool that dumps the filesystem at fs block level, that means that for full and incremental backup the only way to go seems to be tar. This will produce a lot of swapping on disks since tar uses to catalog all the files before writing them. A less hw intensive tool would be cpio but this will not handle incremental backups. I've also searched for utilities dumping LVM2 volumes, or Volume Groups (since our GFS filesystems are on top of them), but nothing exist. If you have some suggestions here, or better alternatives, please let me know. Other than this, I'm trying to figure out which is the best backup plan. I'd go with a weekly full backup for everything (servers local filesystems & GFS volumes) and an incremental daily. We will use 2 different tape sets, to export physically tape cartridges from the data center every week for data safety. What I'm a bit confused about is how to manage the tape cycles during the week, and how to handle backup of big size filesystems (in our case GFS). Actually the filesystem is of around 1TB, but I'm guessing if it would be better to split it in different mountpoints, each of 400GB. This would probably give some benefits for the backup, since this is the exact size of our cartridges. If I'm not wrong, this is what it should be done: Monday 00:00 full servers filesystem dump into 1 cartridge; full GFS tar into multiple tapes OR 1 cartridge for every single 400GB GFS filesystem Tue-Sun 00:00 incremental dump of servers filesystem; incremental GFS tar as above The questions are: - How many cartridges do I need? - How cartridges should be rotated during the week? - Is it possible to use a different set of tape for every week, e.g. two sets to be rotated weekly using a full backup every monday? - Is it better to split the GFS filesystem into multiple 400GB volumes to improve the ease of backup? Sorry for being stressful.. I would hoped to be more specific... Kind regards, Fabio
Re: Need help with backup plan
On Tue, Aug 31, 2004 at 09:20:29PM -0400, Joe Konecny wrote: > > I guess that is my problem. I am currently configured to have > the holding disk on the same physical drive. The best option > appears to be to purchase a new drive. One question though... > I assume the new drive should be at lease as big as the one > I'm backing up? > If you are trying to eliminate the lost data scenario, at least as big as your biggest nightly backup. (times as many days of tapeless backups as you might want to prepare for) -- Jon H. LaBadie [EMAIL PROTECTED] JG Computing 4455 Province Line Road(609) 252-0159 Princeton, NJ 08540-4322 (609) 683-7220 (fax)
Re: Need help with backup plan
Eric Siegerman wrote: There's a little bit of culture clash going on here, I think :-) On Tue, Aug 31, 2004 at 04:29:43PM -0400, Joe Konecny wrote: Even on Sat and Sun when I'm not there the tape is still over written with the latest data. The *last* thing I would *ever* want to do is to overwrite my most recent good backup. What if the current backup fails? [*] Then I've lost them both. The very idea makes me cringe! Given a choice between (a) overwriting last night's backup and (b) failing to take tonight's backup, I'd reluctantly choose the latter. Admittedly, your site's needs might be different from ours, such that a choice that none of us would make, is the right one in your case. But really, I'd much prefer not to have to make that choice at all -- neither (a) nor (b) is very appealing -- and Amanda, used as we've described, gets me out of having to take either risk. [*] System crash, bug in the backup system (Amanda or otherwise), the heads need cleaning, the tape goes bad, the data won't fit on the tape, an attempt to back up a file as it's being written leads to an inconsistent image on the tape, or any of a dozen other failure modes, many of which are orders of magnitude more likely than a disk crash. If Amanda does only copies the data to a holding area and we lose a drive on Sunday night I have massive problems. That's what I call a disaster It's only a disaster if the holding "area" is on the same physical drive as the live data. So don't do that! Besides the robustness problem you've mentioned, it's also bad for backup performance. Put it on its own spindle. I guess that is my problem. I am currently configured to have the holding disk on the same physical drive. The best option appears to be to purchase a new drive. One question though... I assume the new drive should be at lease as big as the one I'm backing up?
Re: Need help with backup plan
There's a little bit of culture clash going on here, I think :-) On Tue, Aug 31, 2004 at 04:29:43PM -0400, Joe Konecny wrote: > Even on Sat and Sun when I'm not > there the tape is still over written with the latest data. The *last* thing I would *ever* want to do is to overwrite my most recent good backup. What if the current backup fails? [*] Then I've lost them both. The very idea makes me cringe! Given a choice between (a) overwriting last night's backup and (b) failing to take tonight's backup, I'd reluctantly choose the latter. Admittedly, your site's needs might be different from ours, such that a choice that none of us would make, is the right one in your case. But really, I'd much prefer not to have to make that choice at all -- neither (a) nor (b) is very appealing -- and Amanda, used as we've described, gets me out of having to take either risk. [*] System crash, bug in the backup system (Amanda or otherwise), the heads need cleaning, the tape goes bad, the data won't fit on the tape, an attempt to back up a file as it's being written leads to an inconsistent image on the tape, or any of a dozen other failure modes, many of which are orders of magnitude more likely than a disk crash. > If Amanda does only copies the data to a holding area and > we lose a drive on Sunday night I have massive problems. > That's what I call a disaster It's only a disaster if the holding "area" is on the same physical drive as the live data. So don't do that! Besides the robustness problem you've mentioned, it's also bad for backup performance. Put it on its own spindle. -- | | /\ |-_|/ > Eric Siegerman, Toronto, Ont.[EMAIL PROTECTED] | | / It must be said that they would have sounded better if the singer wouldn't throw his fellow band members to the ground and toss the drum kit around during songs. - Patrick Lenneau
Re: Need help with backup plan
On Tue, Aug 31, 2004 at 10:42:27PM +0200, Andreas Sundstrom wrote: > I think that if things are so critical I would seriously > investigate in purchasing a tape changer. Indeed! Even better, though perhaps impractical, would be to back the data up across the network to a remote site. Rsync might be a useful tool for this (http://samba.anu.edu.au/rsync/). It's smart enough to only copy the changed portions of a file, so even if your data is a single huge database, rsync might be able to back it up in reasonable time every time but the first. -- | | /\ |-_|/ > Eric Siegerman, Toronto, Ont.[EMAIL PROTECTED] | | / It must be said that they would have sounded better if the singer wouldn't throw his fellow band members to the ground and toss the drum kit around during songs. - Patrick Lenneau
Re: Need help with backup plan
On Tue, Aug 31, 2004 at 04:29:43PM -0400, Joe Konecny wrote: > To me that's ideal since the data collection > in our factory runs 24-7. Even on Sat and Sun when I'm not > there the tape is still over written with the latest data. > If Amanda does only copies the data to a holding area and > we lose a drive on Sunday night I have massive problems. Consider your current Arcserve scheme. If you lose the data-collection drive during a weekend backup run, your recent data is toast. The only backup since Thursday night has been destroyed (partially overwritten) along with the live data. Low probability, but it could happen. Now consider what we're suggesting -- Amanda backing up to holding disk, and the backups being flushed to tape on Monday. To lose the data, you'd have to lose both the live-data disk *and* the holding disk, although the restriction that it has to happen during a backup run doesn't apply. Losing a disk at some point over the weekend is a higher probability than losing it during the backup window, but losing both of them at once is *much less* likely ... unless there's a fire in the machine room or similar catastrophe over the weekend, in which case your current tape backup, which was still in the drive, is toast too -- under either scheme -- and again your most recent backup is the one from Thursday night. > That's what I call a disaster and I don't see how amanda > can help me there. Amanda is approximately as able to help there as your current scheme (equal, or a little better, but unless I'm missing something, not a lot better -- the improvement is precisely the difference in probability of data loss between the two schemes.) Amanda can also guard against failure modes that your current scheme doesn't guard against -- or, as I've described, it can be configured not to guard against them, if that better suits your requirements. -- | | /\ |-_|/ > Eric Siegerman, Toronto, Ont.[EMAIL PROTECTED] | | / It must be said that they would have sounded better if the singer wouldn't throw his fellow band members to the ground and toss the drum kit around during songs. - Patrick Lenneau
Re: Need help with backup plan
On Tue, Aug 31, 2004 at 04:29:43PM -0400, Joe Konecny wrote: > Jon LaBadie wrote: > > >>So... is amanda really not made for disaster recovery? That's great > >>that it will back up to a holding disk with no tape but if the hard > >>drive dies it's rather meaningless. > > > > > >How does that imply "not for disaster recovery"? What would > >your current system do if the tape "jammed"? Would it still > >backup? True, if you encountered a situation where you did > >not put in a tape AND the holding disk died you would not > >have a backup. Is that your concern? > > > > It's because of what I am used to... Right now Arcserve > backs up my Netware server as long as there is a tape in > the drive. To me that's ideal since the data collection > in our factory runs 24-7. Even on Sat and Sun when I'm not > there the tape is still over written with the latest data. > If Amanda does only copies the data to a holding area and > we lose a drive on Sunday night I have massive problems. > That's what I call a disaster and I don't see how amanda > can help me there. I do want to make it work however. > For the disaster you feel is possible several things have to occur at the same time. 1. Your tape or tapedrive fails, or you do not put in a suitable tape - though as has been said by another, with a tapecycle of 1, you could override that protection. 2. A drive containing valuable data would have to fail 3. The drive containing your holding disk would have to fail. Usually this is not the systems primary data drive. And it can also be spread over multiple drives so if one fails others will do the job. What about your current system. Fine, if the wrong tape is inserted you still get a current backup. Same with amanda. It is just on the holding disk. And I'd be worried about something else in your current system. Presently I'm starting to "wear out tapes". Suppose you forgot to change a tape, I'm guessing the tape left in there has the most current backup and is about to be overwritten by the new backup. What if the tape fails, or the backup fails in the middle for some other reason. You will have neither todays nor the last backup. -- Jon H. LaBadie [EMAIL PROTECTED] JG Computing 4455 Province Line Road(609) 252-0159 Princeton, NJ 08540-4322 (609) 683-7220 (fax)
Re: Need help with backup plan
Joe Konecny wrote: Jon LaBadie wrote: So... is amanda really not made for disaster recovery? That's great that it will back up to a holding disk with no tape but if the hard drive dies it's rather meaningless. I don't understand the leap you've made here. The holding disk is a buffer to allow multiple items to be backed up simultaneously but with a single stream to tape. If no "usable" tape is available amanda will still do the backups but leave them in the buffer (holding disk) for later taping. "Usable" here can mean many things including a tape not part of amanda's collection (do you really want it to trash someone else's valuable data?), an amanda tape not permitted to be overwritten (admin controllable), or even a broken drive or no tape inserted. How does that imply "not for disaster recovery"? What would your current system do if the tape "jammed"? Would it still backup? True, if you encountered a situation where you did not put in a tape AND the holding disk died you would not have a backup. Is that your concern? BTW some amanda installations use cheaper, separate drives for the holding disk. On my primary home office system, the 4 system drives are scsi. My holding disk is an IDE drive bigger than the sum of the scsi drives. Once I went on a week long trip forgetting to insert a new set of tapes. Came home, inserted the tapes and flushed a weeks worth of normal backups on to several tapes. It's because of what I am used to... Right now Arcserve backs up my Netware server as long as there is a tape in the drive. To me that's ideal since the data collection in our factory runs 24-7. Even on Sat and Sun when I'm not there the tape is still over written with the latest data. If Amanda does only copies the data to a holding area and we lose a drive on Sunday night I have massive problems. That's what I call a disaster and I don't see how amanda can help me there. I do want to make it work however. I think that if things are so critical I would seriously investigate in purchasing a tape changer.
Re: Need help with backup plan
Jon LaBadie wrote: So... is amanda really not made for disaster recovery? That's great that it will back up to a holding disk with no tape but if the hard drive dies it's rather meaningless. I don't understand the leap you've made here. The holding disk is a buffer to allow multiple items to be backed up simultaneously but with a single stream to tape. If no "usable" tape is available amanda will still do the backups but leave them in the buffer (holding disk) for later taping. "Usable" here can mean many things including a tape not part of amanda's collection (do you really want it to trash someone else's valuable data?), an amanda tape not permitted to be overwritten (admin controllable), or even a broken drive or no tape inserted. How does that imply "not for disaster recovery"? What would your current system do if the tape "jammed"? Would it still backup? True, if you encountered a situation where you did not put in a tape AND the holding disk died you would not have a backup. Is that your concern? BTW some amanda installations use cheaper, separate drives for the holding disk. On my primary home office system, the 4 system drives are scsi. My holding disk is an IDE drive bigger than the sum of the scsi drives. Once I went on a week long trip forgetting to insert a new set of tapes. Came home, inserted the tapes and flushed a weeks worth of normal backups on to several tapes. It's because of what I am used to... Right now Arcserve backs up my Netware server as long as there is a tape in the drive. To me that's ideal since the data collection in our factory runs 24-7. Even on Sat and Sun when I'm not there the tape is still over written with the latest data. If Amanda does only copies the data to a holding area and we lose a drive on Sunday night I have massive problems. That's what I call a disaster and I don't see how amanda can help me there. I do want to make it work however.
Re: Need help with backup plan
Hi, Joe, on Dienstag, 31. August 2004 at 21:49 you wrote to amanda-users: JK> Gavin Henry wrote: >> Amanda is much better than that. It will but the backup in a holding disk >> i.e. folder/partition until space runs out or hits the limit you set. [...] >> Read the docs first and look at this site: >> >> http://www.oops.co.at/AMANDA-docs/index.html Thanks, Gavin. JK> So... is amanda really not made for disaster recovery? That's great JK> that it will back up to a holding disk with no tape but if the hard JK> drive dies it's rather meaningless. Give it a tape. Disaster Recovery is a special thing. AMANDA does still help you in this case, as "she" makes it possible to restore your tapes WITHOUT any AMANDA-software installed. Take any UNIX/Linux with a proper tapedrive and go. It is not a very good argument to say AMANDA is bad, because the disk onto which it dumps your data in the case you failed to change the tape fails. This reminds me of the old problem: You have a light which tells you your brakes failed. What if this light fails? You get a light which tells you that the light failed, that tells you your brakes failed. What if THIS light fails? Dumping to the holdingdisk provides an additional layer of security in the case you put in the wrong tape, your drive eats the tape or something similar. AMANDA is not designed to take care of harddisks. Stefan G. Weichinger mailto:[EMAIL PROTECTED]
Re: Need help with backup plan
Joe Konecny wrote: Gavin Henry wrote: Amanda is much better than that. It will but the backup in a holding disk i.e. folder/partition until space runs out or hits the limit you set. It will not overwrite another tape unless you force it by hand, which is good. You don't have to figure out which tape, amanda tells you. You can even browse the backup files like a normal Unix system and restore per file. You have not spec'd how often you backup either, how much data and much more if we need to help you. Read the docs first and look at this site: http://www.oops.co.at/AMANDA-docs/index.html So... is amanda really not made for disaster recovery? That's great that it will back up to a holding disk with no tape but if the hard drive dies it's rather meaningless. Umm.. You're supposed to flush the holdingdisk to tape as soon as you realise that you forgot to change tapes the other day. Then you are safe..
Re: Need help with backup plan
On Tue, Aug 31, 2004 at 03:49:49PM -0400, Joe Konecny wrote: > Gavin Henry wrote: > > >Amanda is much better than that. It will but the backup in a holding disk > >i.e. folder/partition until space runs out or hits the limit you set. > > > >It will not overwrite another tape unless you force it by hand, which > >is good. > > > >You don't have to figure out which tape, amanda tells you. You can even > >browse the backup files like a normal Unix system and restore per file. > > > >You have not spec'd how often you backup either, how much data and much > >more > >if we need to help you. > > > >Read the docs first and look at this site: > > > >http://www.oops.co.at/AMANDA-docs/index.html > > So... is amanda really not made for disaster recovery? That's great > that it will back up to a holding disk with no tape but if the hard > drive dies it's rather meaningless. I don't understand the leap you've made here. The holding disk is a buffer to allow multiple items to be backed up simultaneously but with a single stream to tape. If no "usable" tape is available amanda will still do the backups but leave them in the buffer (holding disk) for later taping. "Usable" here can mean many things including a tape not part of amanda's collection (do you really want it to trash someone else's valuable data?), an amanda tape not permitted to be overwritten (admin controllable), or even a broken drive or no tape inserted. How does that imply "not for disaster recovery"? What would your current system do if the tape "jammed"? Would it still backup? True, if you encountered a situation where you did not put in a tape AND the holding disk died you would not have a backup. Is that your concern? BTW some amanda installations use cheaper, separate drives for the holding disk. On my primary home office system, the 4 system drives are scsi. My holding disk is an IDE drive bigger than the sum of the scsi drives. Once I went on a week long trip forgetting to insert a new set of tapes. Came home, inserted the tapes and flushed a weeks worth of normal backups on to several tapes. -- Jon H. LaBadie [EMAIL PROTECTED] JG Computing 4455 Province Line Road(609) 252-0159 Princeton, NJ 08540-4322 (609) 683-7220 (fax)
Re: Need help with backup plan
Gavin Henry wrote: Amanda is much better than that. It will but the backup in a holding disk i.e. folder/partition until space runs out or hits the limit you set. It will not overwrite another tape unless you force it by hand, which is good. You don't have to figure out which tape, amanda tells you. You can even browse the backup files like a normal Unix system and restore per file. You have not spec'd how often you backup either, how much data and much more if we need to help you. Read the docs first and look at this site: http://www.oops.co.at/AMANDA-docs/index.html So... is amanda really not made for disaster recovery? That's great that it will back up to a holding disk with no tape but if the hard drive dies it's rather meaningless.
Re: Need help with backup plan
On Tue, Aug 31, 2004 at 10:42:58AM -0400, Joe Konecny wrote: > The system is set up to accept > any tape and run a full backup. What we do is keep the tapes > in rotation and if someone forgets to bring a tape back from > off site it is no big deal we just use the next one. This is the interesting part. Amanda tries to protect you from accidentally overwriting backups that you wanted to keep. To that end, it won't "accept any tape", but only tapes whose contents it considers to have expired. That's what the whole "tapecycle" thing's about. To get what you want, you'll have to defeat this protection. Typically, people have C tapes, and set "tapecycle" to C. Under those conditions (once you've reached equilibrium by running through all the tapes once) Amanda refuses to write to the C-1 most recently used tapes; it will only accept the least recently used tape, C, or a new tape that's never been written to. That's the situation I'm familiar with. But suppose you have more than "tapecycle" tapes, i.e. you have T tapes for some T>C. I believe (please correct me if necessary, folks) that in this case, Amanda will still refuse to write to the C-1 most recently used tapes, but will accept any of the older ones, i.e. any of tapes C through T. It'll say "The next tape Amanda expects to use is: [label for tape T]", but that's not quite true; in fact, it'll accept any tape that isn't in the range [1, C-1]. So, to get the behaviour you have now, I think you can just set "tapecycle" to 1. But I don't know for sure, because I'd never do that :-) Personally, I *like* it that Amanda protects me from my own mistakes (forgetting to change the tape, or mounting the wrong one, for example). Of course, you could use a hybrid approach and get yourself some protection against accidents, but also some of the flexibility you want, simply by setting "tapecycle" to something like 5; then, the latest few tapes would be protected, but any of the older ones would do. > Arcserve > assigns each tape a new ID when it uses it. This, Amanda won't do. You assign each tape an ID (i.e. label) when you amlabel it, and Amanda uses that same label forevermore (unless you manually relabel the tape of course, but Amanda discourages that). That difference in detail shouldn't cause any problem in meeting your basic requirement, though. > We write the date > of the backup on the case of each tape. If a restore is needed > we figure out what date we want and insert the tape. Arcserve > will tell us the tape ID and then we can restore from that > session. In the Amanda world, you wouldn't need to write the backup date on the case. To restore, you'd: - figure out what date you want - use amrecover's "setdate" command, specifying that date - amrecover would tell *you* which tape to mount, asking for it by its label - should you ever switch to doing incremental backups instead of full ones every night, or should your backups grow to more than one tape per run, amrecover might need more than one tape to do a multi-file restore; it'll ask for each one in turn, again by label -- | | /\ |-_|/ > Eric Siegerman, Toronto, Ont.[EMAIL PROTECTED] | | / It must be said that they would have sounded better if the singer wouldn't throw his fellow band members to the ground and toss the drum kit around during songs. - Patrick Lenneau
RE: Need help with backup plan
Amanda is much better than that. It will but the backup in a holding disk i.e. folder/partition until space runs out or hits the limit you set. It will not overwrite another tape unless you force it by hand, which is good. You don't have to figure out which tape, amanda tells you. You can even browse the backup files like a normal Unix system and restore per file. You have not spec'd how often you backup either, how much data and much more if we need to help you. Read the docs first and look at this site: http://www.oops.co.at/AMANDA-docs/index.html -Original Message- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] Behalf Of Joe Konecny Sent: 31 August 2004 15:43 To: [EMAIL PROTECTED] Subject: Need help with backup plan Right now I back up my Netware server with Arcserve to a single tape. I have 10 tapes. The system is set up to accept any tape and run a full backup. What we do is keep the tapes in rotation and if someone forgets to bring a tape back from off site it is no big deal we just use the next one. Arcserve assigns each tape a new ID when it uses it. We write the date of the backup on the case of each tape. If a restore is needed we figure out what date we want and insert the tape. Arcserve will tell us the tape ID and then we can restore from that session. Now I'm switching to FreeBSD and have a working Amanda install with 10 tapes. I have yet to configure the cycle, etc... as I'm not sure how to make it work like Arcserve did. Can Amanda be made to work like that?
Re: Need help with backup plan
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On Tuesday, 31.08.2004 at 10:42 -0400, Joe Konecny wrote: > Right now I back up my Netware server with Arcserve to a single tape. > I have 10 tapes. The system is set up to accept any tape and run a > full backup. What we do is keep the tapes in rotation and if someone > forgets to bring a tape back from off site it is no big deal we just > use the next one. Arcserve assigns each tape a new ID when it uses > it. We write the date of the backup on the case of each tape. If a > restore is needed we figure out what date we want and insert the tape. > Arcserve will tell us the tape ID and then we can restore from that > session. Now I'm switching to FreeBSD and have a working Amanda > install with 10 tapes. I have yet to configure the cycle, etc... as > I'm not sure how to make it work like Arcserve did. Can Amanda be > made to work like that? Won't comment specifically for your case but most questions of "Can Amanda be made to work like that?" are normally not the right thing to ask. Find out how AMANDA actually *does* it and then fit in with its way of doing things. AMANDA is good at scheduling backups. Dave. - -- Dave Ewart [EMAIL PROTECTED] Computing Manager, Epidemiology Unit, Oxford Cancer Research UK PGP: CC70 1883 BD92 E665 B840 118B 6E94 2CFD 694D E370 -BEGIN PGP SIGNATURE- Version: GnuPG v1.2.4 (GNU/Linux) iD8DBQFBNI/fbpQs/WlN43ARAlVzAKC70NcgufKVQtbeK4DLGacf0AMzYgCaA8Gi xZjFI6PMHi4RKbfyZMmf19g= =Qo1J -END PGP SIGNATURE-
Need help with backup plan
Right now I back up my Netware server with Arcserve to a single tape. I have 10 tapes. The system is set up to accept any tape and run a full backup. What we do is keep the tapes in rotation and if someone forgets to bring a tape back from off site it is no big deal we just use the next one. Arcserve assigns each tape a new ID when it uses it. We write the date of the backup on the case of each tape. If a restore is needed we figure out what date we want and insert the tape. Arcserve will tell us the tape ID and then we can restore from that session. Now I'm switching to FreeBSD and have a working Amanda install with 10 tapes. I have yet to configure the cycle, etc... as I'm not sure how to make it work like Arcserve did. Can Amanda be made to work like that?
Re: Backup plan needs advice.
Jon LaBadie wrote:
On Tue, Jun 24, 2003 at 02:25:20PM -0700, bao wrote:
Jon LaBadie wrote:
strategy "string"
...
incronly
Only do incremental dumps. `amadmin force' should
be used to tell Amanda that a full dump has been
performed off-line, so that it resets to level 1.
It is similar to skip-full, but with incronly full
dumps may be scheduled manually. Unfortunately,
it appears that Amanda will perform full backups
with this configuration, which is probably a bug.
Also, another dumptype option entry was this paragraph:
skip-full "boolean"
Default: no. If true and planner has scheduled a full
backup, these disks will be skipped, and full backups
should be run off-line on these days. It was reported
that Amanda only schedules level 1 incrementals in this
configuration; this is probably a bug.
As pointed out. The two paragraphs above have "a bug". I prefer not to
mess with those options. :)
Well, they say "probably". Maybe someone was saying this program/feature
works this way. Probably not only way it could have been designed
and maybe not the best choice for the design -- thus a "bug".
Example, according to the man page for standard unix sort command
has been the line "silently truncates lines longer than bytes".
It could be called a "bug". But in design and implementation,
not a defect in coding.
Plus, the comments may be a holdover from early releases never updated.
There are comments a few years back suggesting the "bug" was fixed.
BTW you could also consider having your "full config" dump to real tape
or to a separate set of "file tapes". Then you could run your daily
config as a normal amanda config doing a mixture of full and incrementals
each day. If you run the full dump to real tape and the clerical person
forgets the tape, it can collect on the holding disk for later amflush.
Otherwise you will have to develop your own procedures for moving the
dumps and the indexes and the ... to tape and later, if needed, you own
procedures for recovering them and the indexes and ... from the tape
for am{recover|restore}.
I have some good news and think I should share it.
With my two configs and with "record yes" in the weekly, the daily
incrementals are able to see
and follow the most recent weekly full backup. That says that Jon's
suggestion of using one config,
inc-only, and forcing a level 0 for the day of the full backup would
also work.
I just tested it with another machine, and will have to wait if anything
unexpected will happen
to the production server.
So going on to investigating how to copy what Amanda needs to tape.
Thanks guys.
Re: Backup plan needs advice.
bao wrote: Hi Martin I forgot to state that this is a tapeless setup to back up to disk. Then the full backup will be transferred to tape, and also is kept on disk for one week. Martin Hepworth wrote: Bao Amanda will need two separate config's to do this, BUT they will have idea of each others existance so the 'daily' will to full backups as and when the amanda planner thinks it's time to do one. I am _not_ letting Amanda schedule as to when to do a full backup, as is recommended. What I want (actually, my boss) is to force it to run a full every Monday, then a level 1 every other day for the rest of the week, according to the most recent Monday. Can't be done. Amanda's planner does this for you based on estimated sizes. I hope the above helps clarify it. Sorry for the confusion. -- Martin Hepworth Senior Systems Administrator Solid State Logic Ltd +44 (0)1865 842300 ** This email and any files transmitted with it are confidential and intended solely for the use of the individual or entity to whom they are addressed. If you have received this email in error please notify the system manager. This footnote also confirms that this email message has been swept by MIMEsweeper for the presence of computer viruses. www.mimesweeper.com **
Re: Backup plan needs advice.
This is getting longer and longer. So, I am going to reply to both Jon's and
Gene's emails in this one email.
-Original Message-
From: Jon LaBadie <[EMAIL PROTECTED]>
To: Amanda Users (E-mail) <[EMAIL PROTECTED]>
Date: Tuesday, June 24, 2003 2:57 PM
Subject: Re: Backup plan needs advice.
>On Tue, Jun 24, 2003 at 02:25:20PM -0700, bao wrote:
>>
>> Jon LaBadie wrote:
>>
>> >strategy "string"
>> >
>> > ...
>> > incronly
>> > Only do incremental dumps. `amadmin force' should
>> > be used to tell Amanda that a full dump has been
>> > performed off-line, so that it resets to level 1.
>> > It is similar to skip-full, but with incronly full
>> > dumps may be scheduled manually. Unfortunately,
>> > it appears that Amanda will perform full backups
>> > with this configuration, which is probably a bug.
>> >
>> >
>> >Also, another dumptype option entry was this paragraph:
>> >
>> >skip-full "boolean"
>> > Default: no. If true and planner has scheduled a full
>> > backup, these disks will be skipped, and full backups
>> > should be run off-line on these days. It was reported
>> > that Amanda only schedules level 1 incrementals in this
>> > configuration; this is probably a bug.
>> >
>> >
>> >
>> As pointed out. The two paragraphs above have "a bug". I prefer not to
>> mess with those options. :)
>
>Well, they say "probably". Maybe someone was saying this program/feature
>works this way. Probably not only way it could have been designed
>and maybe not the best choice for the design -- thus a "bug".
>Example, according to the man page for standard unix sort command
>has been the line "silently truncates lines longer than bytes".
>It could be called a "bug". But in design and implementation,
>not a defect in coding.
>
>
>Plus, the comments may be a holdover from early releases never updated.
>There are comments a few years back suggesting the "bug" was fixed.
>
>
>BTW you could also consider having your "full config" dump to real tape
>or to a separate set of "file tapes". Then you could run your daily
>config as a normal amanda config doing a mixture of full and incrementals
>each day. If you run the full dump to real tape and the clerical person
>forgets the tape, it can collect on the holding disk for later amflush.
>Otherwise you will have to develop your own procedures for moving the
>dumps and the indexes and the ... to tape and later, if needed, you own
>procedures for recovering them and the indexes and ... from the tape
>for am{recover|restore}.
>
As I have two configs and two sets of "tapes" now, but they behave as just
one continuous set of tapes.
If I do as Jon said, dump the full to a real tape, and the inc to a set of
"tapes", then the full to tape and
the full of the daily config probably will not be consistent, as they will
have to be run at different times.
I think someone had given his/her instructions of how to copy the backup
image and the index some while back
(maybe Gene) to tape. I will have to dig more into my collection of mail
from this group to look for it.
I am still leaving it to run with my original two configs and waiting to see
what will happen in the next few days
as the new cycle begins. If it doesn't work out, I will look into what Jon
suggested, one config.
I'm still facing the problem of verifing if the next first incremental will
follow the next full, or the full from the previous
week. Please give me any suggestions you can think of to accomplish this.
Gene, diskspace is not a problem to us. Our data consists of only around 20
GB. We have _plenty_ plenty of hard disk
space (it's very cheap nowadays). Our tape is 20 GB native, expanding to 40
GB with compression.
It's been a very helpful and fun discussion. Many thanks to both Jon and
Gene.
I hope there will be follow-ups.
Cheers.
>--
>Jon H. LaBadie [EMAIL PROTECTED]
> JG Computing
> 4455 Province Line Road(609) 252-0159
> Princeton, NJ 08540-4322 (609) 683-7220 (fax)
>
Re: Backup plan needs advice.
On Tuesday 24 June 2003 17:14, bao wrote: >Sorry I should have answered your other questions in the same email. > But I just...saw them :) > >Gene Heskett wrote: >>On Tuesday 24 June 2003 12:38, bao wrote: >>>Hello, >>> >>>After a very long and hard time struggling with Amanda, I figured >>>out that it's the Athlon bug, and have >>>switched to an Intel. Now it works. >> >>The Athlon Bug? Sorry, please elucidate on this subject, its >> entirely new to me as I have been running amanda on an Athlon >> DX1600 for 2 years with no problems I didn't create. Likewise for >> around 3 years on a K6-II and then a K6-III. >> >>>However, I am still confused of >>>some things regarding the backup >>>plan, and hope someone will help me answer them. >>> >>>I would like to back up everyday, in incremental, and a weekly >>> full backup once a week, on a specified >>>day. Let the full be on Monday, and the incrementals be on the >>> rest of the week. I now have to configs: >> >>I believe the word is two? >> >>>1. Weekly: has dumpcycle=0, runspercycle=1, tapecycle=1, >>>strategy=noinc, index=yes, and record=no >>>2. Daily: has dumpcycle=7, runspercycle=6, tapecycle=6, >>>strategy=nofull, , index=yes, record=yes >>> >>>The Weekly full will call amadmin to set to run level 0 before it >>>runs. >>> >>>Q1: If I want the incrementals of a week to follow from that >>> week's full, how do I set the record?? >>>(Mon : level 0, Tues-Sun : level 1 - backing up any changes from >>> the most recent Monday) >>> >>>Q2. The Weekly full backup will be transferred to tape with a >>> cycle of 8 tapes, then rotate. What do >>>I need to copy to tape beside the backup image?? index ?? log ?? >>>debug files ?? >>> >>>Thank you all, >> >>Me, gets up on shaky old soapbox. >> >>While amanda can be forced to do it that way, thats not how amanda >> was designed to run. > >I guess, _whispering_, that my boss is too cheap to get a commercial >product that suits >his needs. Everyone has been asking me, "Why not let Amanda do it > her way?". My answer is: "My boss says, 'Why not let it work MY > way??' " I give up. ;) He should price knox's arkeia, or veritas, heck even bru. I think any of those would be easier to make fit that mold. And obviously the 'Boss' has *his* way carved in granite. But that wouldn't stop me from chipping away at it every chance I had. :) >>Whats wrong with letting amanda do it her way? > >Letting Amanda do it her way will scatter the full backup out to > many tapes (I believe). This is true. But you could point out that if one tape is lost, everything is not lost like it would be if something should fubar the full backup he wants. >And yes, we want a bulky full backup each week, then many small >incremantals during the >weekdays. We don't mind about inefficient usage of tape. The weekly > will stay on disk, and >also go to tape. BTW, that weekly copy will be erased as the tapes >rotate. The incrementals >stay on disk for quick recovery from disk instead of from slow tape. >The weekly stays on disk for the same purpose. > >>The result is the same in that you'll have a good backup strategy [...] >One reason for doing it this way is: If the full runs on a holiday > when no one is in, >it is left on the disk and we have six full days to flush it to tape > by hand any time we >want, by any way we like. If there is sufficient disk capacity, then this should work just fine. >We have our secretary as the tape changer :) >She has very little thing to do, I believe. Don't forget that some think the maintainace of a shine free nose is at least as important as sending out the monthly billing. I've suffered thru a couple of those in my 68 years... I've also had the displeasure of one that thought if her hands weren't dripping with grease from a hand lotion, it was time to apply some more of it. Keep that type away from your tapes, at gunpoint if required. [...] -- Cheers, Gene AMD [EMAIL PROTECTED] 320M [EMAIL PROTECTED] 512M 99.26% setiathome rank, not too shabby for a WV hillbilly Yahoo.com attornies please note, additions to this message by Gene Heskett are: Copyright 2003 by Maurice Eugene Heskett, all rights reserved.
Re: Backup plan needs advice.
On Tuesday 24 June 2003 17:01, bao wrote: >Gene Heskett wrote: >>On Tuesday 24 June 2003 12:38, bao wrote: >>>Hello, >>> >>>After a very long and hard time struggling with Amanda, I figured >>>out that it's the Athlon bug, and have >>>switched to an Intel. Now it works. >> >>The Athlon Bug? Sorry, please elucidate on this subject, its >> entirely new to me as I have been running amanda on an Athlon >> DX1600 for 2 years with no problems I didn't create. Likewise for >> around 3 years on a K6-II and then a K6-III. > >This subject, I believed, emerged within a year and a half. It is >related to the 4MB page >size that is the default for Linux compiled on an Intel processor, > but run on Athlon. >This is the one that we had been experiencing. This caused a kernel >halt/crash/reboot instantly >when I ran, first with Tapeware, then with Amanda, and finally with > tar. It is said to be fixed >in new kernels, but it still does not work for our Athlon. Too > tired, we got a new Pentium 4. >Cool heh ?? :) Maybe, but then I haven't run the stock kernel except under extreme duress in 5 years. I build my own, with the appropriate flags for an athlon (or a K6) set in the .config. Currently running a 2.4.21 kernel. >For more info, go to this link >http://www.geocrawler.com/lists/3/Linux/35/175/7626960/ > >>>However, I am still confused of >>>some things regarding the backup >>>plan, and hope someone will help me answer them. >>> >>>I would like to back up everyday, in incremental, and a weekly >>> full backup once a week, on a specified >>>day. Let the full be on Monday, and the incrementals be on the >>> rest of the week. I now have to configs: >> >>I believe the word is two? >> >>>1. Weekly: has dumpcycle=0, runspercycle=1, tapecycle=1, >>>strategy=noinc, index=yes, and record=no >>>2. Daily: has dumpcycle=7, runspercycle=6, tapecycle=6, >>>strategy=nofull, , index=yes, record=yes >>> >>>The Weekly full will call amadmin to set to run level 0 before it >>>runs. >>> >>>Q1: If I want the incrementals of a week to follow from that >>> week's full, how do I set the record?? >>>(Mon : level 0, Tues-Sun : level 1 - backing up any changes from >>> the most recent Monday) >>> >>>Q2. The Weekly full backup will be transferred to tape with a >>> cycle of 8 tapes, then rotate. What do >>>I need to copy to tape beside the backup image?? index ?? log ?? >>>debug files ?? Both Q's are probably better answered by someone who has actually done it. I have not. What I'm presently finetuning is a script that runs after amdump is done, that appends the indices generated by the making of *that* tape, and the amanda config that generated that tape, something that is always one tape behind when done by amanda herself. >>>Thank you all, >> [snip sermon :-)] -- Cheers, Gene AMD [EMAIL PROTECTED] 320M [EMAIL PROTECTED] 512M 99.26% setiathome rank, not too shabby for a WV hillbilly Yahoo.com attornies please note, additions to this message by Gene Heskett are: Copyright 2003 by Maurice Eugene Heskett, all rights reserved.
Re: Backup plan needs advice.
On Tue, Jun 24, 2003 at 02:25:20PM -0700, bao wrote:
>
> Jon LaBadie wrote:
>
> >strategy "string"
> >
> > ...
> > incronly
> > Only do incremental dumps. `amadmin force' should
> > be used to tell Amanda that a full dump has been
> > performed off-line, so that it resets to level 1.
> > It is similar to skip-full, but with incronly full
> > dumps may be scheduled manually. Unfortunately,
> > it appears that Amanda will perform full backups
> > with this configuration, which is probably a bug.
> >
> >
> >Also, another dumptype option entry was this paragraph:
> >
> >skip-full "boolean"
> > Default: no. If true and planner has scheduled a full
> > backup, these disks will be skipped, and full backups
> > should be run off-line on these days. It was reported
> > that Amanda only schedules level 1 incrementals in this
> > configuration; this is probably a bug.
> >
> >
> >
> As pointed out. The two paragraphs above have "a bug". I prefer not to
> mess with those options. :)
Well, they say "probably". Maybe someone was saying this program/feature
works this way. Probably not only way it could have been designed
and maybe not the best choice for the design -- thus a "bug".
Example, according to the man page for standard unix sort command
has been the line "silently truncates lines longer than bytes".
It could be called a "bug". But in design and implementation,
not a defect in coding.
Plus, the comments may be a holdover from early releases never updated.
There are comments a few years back suggesting the "bug" was fixed.
BTW you could also consider having your "full config" dump to real tape
or to a separate set of "file tapes". Then you could run your daily
config as a normal amanda config doing a mixture of full and incrementals
each day. If you run the full dump to real tape and the clerical person
forgets the tape, it can collect on the holding disk for later amflush.
Otherwise you will have to develop your own procedures for moving the
dumps and the indexes and the ... to tape and later, if needed, you own
procedures for recovering them and the indexes and ... from the tape
for am{recover|restore}.
--
Jon H. LaBadie [EMAIL PROTECTED]
JG Computing
4455 Province Line Road(609) 252-0159
Princeton, NJ 08540-4322 (609) 683-7220 (fax)
Re: Backup plan needs advice.
Jon LaBadie wrote: On Tue, Jun 24, 2003 at 11:05:00AM -0700, bao wrote: Jon LaBadie wrote: On Tue, Jun 24, 2003 at 10:12:01AM -0700, bao wrote: I forgot to state that this is a tapeless setup to back up to disk. Then the full backup will be transferred to tape, and also is kept on disk for one week. I am _not_ letting Amanda schedule as to when to do a full backup, as is recommended. What I want (actually, my boss) is to force it to run a full every Monday, then a level 1 every other day for the rest of the week, according to the most recent Monday. I'm not familiar with the various "strategy" and other related options, but can you set a strategy that always does a "no-full" or "only-incremental" AND can also be forced with amadmin to do full dumps on demand? Jon, I am reluctant to try that, although I agree that it is a good candidate. This is because we had to do a recover yesterday, when I didn't think that the system was ready for it. Such a relief that it worked, because a major part of the data was deleted accidentally :) I don't get the reluctance. You don't mean you plan to put this into effect as the actual one and only live backup system without some testing do you? :) Besides, my idea, if it works, doesn't eliminate the ability to have a separate FULL ONLY config. Though in that case I'd probably us a different set of "file tapes" and "record no". It just happened that our backup system, Tapeware, didn't function properly yesterday. It would freeze, but luckily not the whole system. Amanda was still in testing phase, but had been pulled out for recovery. There is hardly a way out from this mess anymore. Back to my question, if I configure it as you suggested, would the incrementals have any idea of the previous forced full dump, and continue on from that point? Or would they go on after the ...very first full backup?? Well I wondered about the possibilities some more, and as I had a file:driver setup for test purposes (good way to check out exclude/include and other small changes) I did a little reading. Yes Gene, there goes Jon again, reading the docs :) Lo and Behold, on the amanda man page of all places: strategy "string" ... nofull Never do full backups, only level 1 incrementals. ... incronly Only do incremental dumps. `amadmin force' should be used to tell Amanda that a full dump has been performed off-line, so that it resets to level 1. It is similar to skip-full, but with incronly full dumps may be scheduled manually. Unfortunately, it appears that Amanda will perform full backups with this configuration, which is probably a bug. Also, another dumptype option entry was this paragraph: skip-full "boolean" Default: no. If true and planner has scheduled a full backup, these disks will be skipped, and full backups should be run off-line on these days. It was reported that Amanda only schedules level 1 incrementals in this configuration; this is probably a bug. As pointed out. The two paragraphs above have "a bug". I prefer not to mess with those options. :) Not knowing which set of options had a good chance to work, I set my dumpcycle and runspercycle to 1 day/1 and my strategy to "nofull". I was very surprised when I ran my first amdump and got a full level 0. As this was the first run for the disk, a level 0 should have been scheduled, but I expected the "nofull" strategy to supercede and cause an error. Six subsequent amdumps were all incrementals. I then set my dumpcycle to 0 days, normally something that would force a level 0 each run. Each of the next two amdumps each generated level 0's. If yes, perhaps you can use just a single configuration. The incremental only strategy would only cause incrementals daily except the one following your forced "amadmin full". To me, it seems a single config would work. But you may have to experiment with the dumptype options skip-full, strategy "nofull", and strategy "incronly" to decide which is the proper approach. Those options seem to be a real muddy area in amanda. If the answer to the question above is yes, then it would also work with two configurations No, I don't think so, because in a config, you must do a full before you can do ANY incrementals. Perhaps you can get the ball rolling with one forced level 0, I don't know. I think you're right. I ran a full backup last Thu. The next day, I wanted to run an incremental following the full, but it ran as level 0. The subsequent days it ran with level 1. This Thu, I will run another full. I am worried about the incremental this Fri. I don't know if it will follow the full this coming Thu, or the last Thu. Is there any way to check?? (and if the incrementals have an idea about the full). If so, the n
Re: Backup plan needs advice.
Sorry I should have answered your other questions in the same email. But
I just...saw them :)
Gene Heskett wrote:
On Tuesday 24 June 2003 12:38, bao wrote:
Hello,
After a very long and hard time struggling with Amanda, I figured
out that it's the Athlon bug, and have
switched to an Intel. Now it works.
The Athlon Bug? Sorry, please elucidate on this subject, its entirely
new to me as I have been running amanda on an Athlon DX1600 for 2
years with no problems I didn't create. Likewise for around 3 years
on a K6-II and then a K6-III.
However, I am still confused of
some things regarding the backup
plan, and hope someone will help me answer them.
I would like to back up everyday, in incremental, and a weekly full
backup once a week, on a specified
day. Let the full be on Monday, and the incrementals be on the rest
of the week. I now have to configs:
I believe the word is two?
1. Weekly: has dumpcycle=0, runspercycle=1, tapecycle=1,
strategy=noinc, index=yes, and record=no
2. Daily: has dumpcycle=7, runspercycle=6, tapecycle=6,
strategy=nofull, , index=yes, record=yes
The Weekly full will call amadmin to set to run level 0 before it
runs.
Q1: If I want the incrementals of a week to follow from that week's
full, how do I set the record??
(Mon : level 0, Tues-Sun : level 1 - backing up any changes from the
most recent Monday)
Q2. The Weekly full backup will be transferred to tape with a cycle
of 8 tapes, then rotate. What do
I need to copy to tape beside the backup image?? index ?? log ??
debug files ??
Thank you all,
Me, gets up on shaky old soapbox.
While amanda can be forced to do it that way, thats not how amanda was
designed to run.
I guess, _whispering_, that my boss is too cheap to get a commercial
product that suits
his needs. Everyone has been asking me, "Why not let Amanda do it her way?".
My answer is: "My boss says, 'Why not let it work MY way??' "
I give up. ;)
Whats wrong with letting amanda do it her way?
Letting Amanda do it her way will scatter the full backup out to many
tapes (I believe).
And yes, we want a bulky full backup each week, then many small
incremantals during the
weekdays. We don't mind about inefficient usage of tape. The weekly will
stay on disk, and
also go to tape. BTW, that weekly copy will be erased as the tapes
rotate. The incrementals
stay on disk for quick recovery from disk instead of from slow tape.
The weekly stays on disk for the same purpose.
The result is the same in that you'll have a good backup strategy.
Doing it amanda's way will result in similar amounts of data being
backed up up each night because amanda will adjust the schedule over
many runs trying to do just that.
Doing it your way will get you one huge backup you'll have to babysit
if you don't have a changer, and very little tape useage during the
week, but the tapes will still be used. Doing it amanda's way you
can use a much smaller tape, and use about 90% of it every night.
One reason for doing it this way is: If the full runs on a holiday when
no one is in,
it is left on the disk and we have six full days to flush it to tape by
hand any time we
want, by any way we like.
We have our secretary as the tape changer :)
She has very little thing to do, I believe.
Somehow, that seems to be the more efficient method of the two.
I'm a home user, backing up just two machines, with a total drive
capacity of about 170Gb, but only maybe 65Gb in the disklist, with a
dumpcycle=7, runspercycle=7, tapecycle=28, runtapes=1, and doing it
to 4Gb uncompressed DDS2 tapes.
To demo how this works, from last nights email:
---
These dumps were to tape DailySet1-02.
The next tape Amanda expects to use is: DailySet1-03.
STATISTICS:
Total Full Daily
Estimate Time (hrs:min)0:19
Run Time (hrs:min) 2:54
Dump Time (hrs:min)0:29 0:05 0:24
Output Size (meg)3551.3 3168.6 382.7
Original Size (meg) 4025.1 3168.6 856.5
Avg Compressed Size (%)38.2--38.2 (level:#disks
...)
Filesystems Dumped 44 6 38 (1:35 2:3)
Avg Dump Rate (k/s) 2055.810628.0 267.8
Tape Time (hrs:min)2:34 2:17 0:17
Tape Size (meg) 3551.3 3168.6 382.7
Tape Used (%) 96.6 86.2 10.4 (level:#disks
...)
Filesystems Taped44 6 38 (1:35 2:3)
Avg Tp Write Rate (k/s) 393.8 393.9 392.7
USAGE BY TAPE:
Label Time Size %Nb
DailySet1-02 2:343551.3 96.644
Thats not the whole email from amanda of course, but it shows several
things.
First, last nights run apparently covered mostly level 0 DLE's that
aren't compressed or can't be compres
Re: Backup plan needs advice.
Gene Heskett wrote:
On Tuesday 24 June 2003 12:38, bao wrote:
Hello,
After a very long and hard time struggling with Amanda, I figured
out that it's the Athlon bug, and have
switched to an Intel. Now it works.
The Athlon Bug? Sorry, please elucidate on this subject, its entirely
new to me as I have been running amanda on an Athlon DX1600 for 2
years with no problems I didn't create. Likewise for around 3 years
on a K6-II and then a K6-III.
This subject, I believed, emerged within a year and a half. It is
related to the 4MB page
size that is the default for Linux compiled on an Intel processor, but
run on Athlon.
This is the one that we had been experiencing. This caused a kernel
halt/crash/reboot instantly
when I ran, first with Tapeware, then with Amanda, and finally with tar.
It is said to be fixed
in new kernels, but it still does not work for our Athlon. Too tired, we
got a new Pentium 4.
Cool heh ?? :)
For more info, go to this link
http://www.geocrawler.com/lists/3/Linux/35/175/7626960/
However, I am still confused of
some things regarding the backup
plan, and hope someone will help me answer them.
I would like to back up everyday, in incremental, and a weekly full
backup once a week, on a specified
day. Let the full be on Monday, and the incrementals be on the rest
of the week. I now have to configs:
I believe the word is two?
1. Weekly: has dumpcycle=0, runspercycle=1, tapecycle=1,
strategy=noinc, index=yes, and record=no
2. Daily: has dumpcycle=7, runspercycle=6, tapecycle=6,
strategy=nofull, , index=yes, record=yes
The Weekly full will call amadmin to set to run level 0 before it
runs.
Q1: If I want the incrementals of a week to follow from that week's
full, how do I set the record??
(Mon : level 0, Tues-Sun : level 1 - backing up any changes from the
most recent Monday)
Q2. The Weekly full backup will be transferred to tape with a cycle
of 8 tapes, then rotate. What do
I need to copy to tape beside the backup image?? index ?? log ??
debug files ??
Thank you all,
Me, gets up on shaky old soapbox.
While amanda can be forced to do it that way, thats not how amanda was
designed to run.
Whats wrong with letting amanda do it her way?
The result is the same in that you'll have a good backup strategy.
Doing it amanda's way will result in similar amounts of data being
backed up up each night because amanda will adjust the schedule over
many runs trying to do just that.
Doing it your way will get you one huge backup you'll have to babysit
if you don't have a changer, and very little tape useage during the
week, but the tapes will still be used. Doing it amanda's way you
can use a much smaller tape, and use about 90% of it every night.
Somehow, that seems to be the more efficient method of the two.
I'm a home user, backing up just two machines, with a total drive
capacity of about 170Gb, but only maybe 65Gb in the disklist, with a
dumpcycle=7, runspercycle=7, tapecycle=28, runtapes=1, and doing it
to 4Gb uncompressed DDS2 tapes.
To demo how this works, from last nights email:
---
These dumps were to tape DailySet1-02.
The next tape Amanda expects to use is: DailySet1-03.
STATISTICS:
Total Full Daily
Estimate Time (hrs:min)0:19
Run Time (hrs:min) 2:54
Dump Time (hrs:min)0:29 0:05 0:24
Output Size (meg)3551.3 3168.6 382.7
Original Size (meg) 4025.1 3168.6 856.5
Avg Compressed Size (%)38.2--38.2 (level:#disks
...)
Filesystems Dumped 44 6 38 (1:35 2:3)
Avg Dump Rate (k/s) 2055.810628.0 267.8
Tape Time (hrs:min)2:34 2:17 0:17
Tape Size (meg) 3551.3 3168.6 382.7
Tape Used (%) 96.6 86.2 10.4 (level:#disks
...)
Filesystems Taped44 6 38 (1:35 2:3)
Avg Tp Write Rate (k/s) 393.8 393.9 392.7
USAGE BY TAPE:
Label Time Size %Nb
DailySet1-02 2:343551.3 96.644
Thats not the whole email from amanda of course, but it shows several
things.
First, last nights run apparently covered mostly level 0 DLE's that
aren't compressed or can't be compressed. When the opposite is true,
I have seen the Original size line display figures above 10Gb. From
the June 3rd run for instance:
Output Size (meg)3560.0 3416.8 143.1
Original Size (meg) 10003.3 9746.4 257.0
Second, that amanda used what it thought was 96.6% of the tapes
capacity, which I have reduced slightly in the tapetype in order to
save room for about 100 megs of stuff I append to the end of the tape
after amanda is done.
Third, that those DLE's which are set to be compressed, compressed to
38.2% of their original size. It
Re: Backup plan needs advice.
On Tuesday 24 June 2003 12:38, bao wrote:
>Hello,
>
>After a very long and hard time struggling with Amanda, I figured
> out that it's the Athlon bug, and have
>switched to an Intel. Now it works.
The Athlon Bug? Sorry, please elucidate on this subject, its entirely
new to me as I have been running amanda on an Athlon DX1600 for 2
years with no problems I didn't create. Likewise for around 3 years
on a K6-II and then a K6-III.
>However, I am still confused of
> some things regarding the backup
>plan, and hope someone will help me answer them.
>
>I would like to back up everyday, in incremental, and a weekly full
>backup once a week, on a specified
>day. Let the full be on Monday, and the incrementals be on the rest
> of the week. I now have to configs:
I believe the word is two?
>1. Weekly: has dumpcycle=0, runspercycle=1, tapecycle=1,
> strategy=noinc, index=yes, and record=no
>2. Daily: has dumpcycle=7, runspercycle=6, tapecycle=6,
>strategy=nofull, , index=yes, record=yes
>
>The Weekly full will call amadmin to set to run level 0 before it
> runs.
>
>Q1: If I want the incrementals of a week to follow from that week's
>full, how do I set the record??
>(Mon : level 0, Tues-Sun : level 1 - backing up any changes from the
>most recent Monday)
>
>Q2. The Weekly full backup will be transferred to tape with a cycle
> of 8 tapes, then rotate. What do
>I need to copy to tape beside the backup image?? index ?? log ??
> debug files ??
>
>Thank you all,
Me, gets up on shaky old soapbox.
While amanda can be forced to do it that way, thats not how amanda was
designed to run.
Whats wrong with letting amanda do it her way?
The result is the same in that you'll have a good backup strategy.
Doing it amanda's way will result in similar amounts of data being
backed up up each night because amanda will adjust the schedule over
many runs trying to do just that.
Doing it your way will get you one huge backup you'll have to babysit
if you don't have a changer, and very little tape useage during the
week, but the tapes will still be used. Doing it amanda's way you
can use a much smaller tape, and use about 90% of it every night.
Somehow, that seems to be the more efficient method of the two.
I'm a home user, backing up just two machines, with a total drive
capacity of about 170Gb, but only maybe 65Gb in the disklist, with a
dumpcycle=7, runspercycle=7, tapecycle=28, runtapes=1, and doing it
to 4Gb uncompressed DDS2 tapes.
To demo how this works, from last nights email:
---
These dumps were to tape DailySet1-02.
The next tape Amanda expects to use is: DailySet1-03.
STATISTICS:
Total Full Daily
Estimate Time (hrs:min)0:19
Run Time (hrs:min) 2:54
Dump Time (hrs:min)0:29 0:05 0:24
Output Size (meg)3551.3 3168.6 382.7
Original Size (meg) 4025.1 3168.6 856.5
Avg Compressed Size (%)38.2--38.2 (level:#disks
...)
Filesystems Dumped 44 6 38 (1:35 2:3)
Avg Dump Rate (k/s) 2055.810628.0 267.8
Tape Time (hrs:min)2:34 2:17 0:17
Tape Size (meg) 3551.3 3168.6 382.7
Tape Used (%) 96.6 86.2 10.4 (level:#disks
...)
Filesystems Taped44 6 38 (1:35 2:3)
Avg Tp Write Rate (k/s) 393.8 393.9 392.7
USAGE BY TAPE:
Label Time Size %Nb
DailySet1-02 2:343551.3 96.644
Thats not the whole email from amanda of course, but it shows several
things.
First, last nights run apparently covered mostly level 0 DLE's that
aren't compressed or can't be compressed. When the opposite is true,
I have seen the Original size line display figures above 10Gb. From
the June 3rd run for instance:
Output Size (meg)3560.0 3416.8 143.1
Original Size (meg) 10003.3 9746.4 257.0
Second, that amanda used what it thought was 96.6% of the tapes
capacity, which I have reduced slightly in the tapetype in order to
save room for about 100 megs of stuff I append to the end of the tape
after amanda is done.
Third, that those DLE's which are set to be compressed, compressed to
38.2% of their original size. It will vary some, but thats typical,
and it beats the socks off of useing hardware compression. Those
DLE's I don't compress cannot be compressed by the hardware
compressor either, and may in fact grow some.
Me, falls off collapsing soapbox, skins elbow. ;-)
--
Cheers, Gene
AMD [EMAIL PROTECTED] 320M
[EMAIL PROTECTED] 512M
99.26% setiathome rank, not too shabby for a WV hillbilly
Yahoo.com attornies please note, additions to this message
by Gene Heskett are:
Copyright 2003 by Maurice Eugene Heskett, all rights reserved.
Re: Backup plan needs advice.
On Tue, Jun 24, 2003 at 11:05:00AM -0700, bao wrote: > Jon LaBadie wrote: > >On Tue, Jun 24, 2003 at 10:12:01AM -0700, bao wrote: > >> > >>I forgot to state that this is a tapeless setup to back up to disk. > >>Then the full backup will be transferred to tape, and also is kept > >> on disk for one week. > >> > >>I am _not_ letting Amanda schedule as to when to do a full backup, > >>as is recommended. What I want (actually, my boss) is to force > >>it to run a full every Monday, then a level 1 every other day for > >>the rest of the week, according to the most recent Monday. > >> > > > >I'm not familiar with the various "strategy" and other related > >options, but can you set a strategy that always does a "no-full" > >or "only-incremental" AND can also be forced with amadmin to do > >full dumps on demand? > > > Jon, I am reluctant to try that, although I agree that it is > a good candidate. This is because we had to do a recover > yesterday, when I didn't think that the system was ready > for it. Such a relief that it worked, because a major part > of the data was deleted accidentally :) I don't get the reluctance. You don't mean you plan to put this into effect as the actual one and only live backup system without some testing do you? :) Besides, my idea, if it works, doesn't eliminate the ability to have a separate FULL ONLY config. Though in that case I'd probably us a different set of "file tapes" and "record no". > Back to my question, if I configure it as you suggested, would > the incrementals have any idea of the previous forced full dump, > and continue on from that point? Or would they go on after the > ...very first full backup?? Well I wondered about the possibilities some more, and as I had a file:driver setup for test purposes (good way to check out exclude/include and other small changes) I did a little reading. Yes Gene, there goes Jon again, reading the docs :) Lo and Behold, on the amanda man page of all places: strategy "string" ... nofull Never do full backups, only level 1 incrementals. ... incronly Only do incremental dumps. `amadmin force' should be used to tell Amanda that a full dump has been performed off-line, so that it resets to level 1. It is similar to skip-full, but with incronly full dumps may be scheduled manually. Unfortunately, it appears that Amanda will perform full backups with this configuration, which is probably a bug. Also, another dumptype option entry was this paragraph: skip-full "boolean" Default: no. If true and planner has scheduled a full backup, these disks will be skipped, and full backups should be run off-line on these days. It was reported that Amanda only schedules level 1 incrementals in this configuration; this is probably a bug. Not knowing which set of options had a good chance to work, I set my dumpcycle and runspercycle to 1 day/1 and my strategy to "nofull". I was very surprised when I ran my first amdump and got a full level 0. As this was the first run for the disk, a level 0 should have been scheduled, but I expected the "nofull" strategy to supercede and cause an error. Six subsequent amdumps were all incrementals. I then set my dumpcycle to 0 days, normally something that would force a level 0 each run. Each of the next two amdumps each generated level 0's. > > >If yes, perhaps you can use just a single configuration. > >The incremental only strategy would only cause incrementals daily > >except the one following your forced "amadmin full". > > To me, it seems a single config would work. But you may have to experiment with the dumptype options skip-full, strategy "nofull", and strategy "incronly" to decide which is the proper approach. Those options seem to be a real muddy area in amanda. > If the answer to the question above is yes, then it would also work with > two configurations No, I don't think so, because in a config, you must do a full before you can do ANY incrementals. Perhaps you can get the ball rolling with one forced level 0, I don't know. > (and if the incrementals have an idea about the full). If so, the next > question arises is do I need to have "record yes" in the full config, > in order for the incrementals to know?? The setting of record would definitely affect what files the incrementals consider changed. Without a record "yes", the incrementals would still consider the last level 0, the last "recorded" one. -- Jon H. LaBadie [EMAIL PROTECTED] JG Computing 4455 Province Line Road(609) 252-0159 Princeton, NJ 08540-4322 (609) 683-7220 (fax)
Re: Backup plan needs advice.
Jon LaBadie wrote: On Tue, Jun 24, 2003 at 10:12:01AM -0700, bao wrote: Hi Martin I forgot to state that this is a tapeless setup to back up to disk. Then the full backup will be transferred to tape, and also is kept on disk for one week. I am _not_ letting Amanda schedule as to when to do a full backup, as is recommended. What I want (actually, my boss) is to force it to run a full every Monday, then a level 1 every other day for the rest of the week, according to the most recent Monday. I hope the above helps clarify it. I'm not familiar with the various "strategy" and other related options, but can you set a strategy that always does a "no-full" or "only-incremental" AND can also be forced with amadmin to do full dumps on demand? Jon, I am reluctant to try that, although I agree that it is a good candidate. This is because we had to do a recover yesterday, when I didn't think that the system was ready for it. Such a relief that it worked, because a major part of the data was deleted accidentally :) Back to my question, if I configure it as you suggested, would the incrementals have any idea of the previous forced full dump, and continue on from that point? Or would they go on after the ...very first full backup?? If yes, perhaps you can use just a single configuration. The incremental only strategy would only cause incrementals daily except the one following your forced "amadmin full" Then dump that "file tape" to real tape. If the answer to the question above is yes, then it would also work with two configurations (and if the incrementals have an idea about the full). If so, the next question arises is do I need to have "record yes" in the full config, in order for the incrementals to know?? Thanks Jon.
Re: Backup plan needs advice.
On Tue, Jun 24, 2003 at 10:12:01AM -0700, bao wrote: > Hi Martin > > I forgot to state that this is a tapeless setup to back up to disk. Then > the full backup will be transferred to tape, > and also is kept on disk for one week. > > > > > I am _not_ letting Amanda schedule as to when to do a full backup, as is > recommended. What I want > (actually, my boss) is to force it to run a full every Monday, then a > level 1 every other day for the rest of > the week, according to the most recent Monday. > > I hope the above helps clarify it. I'm not familiar with the various "strategy" and other related options, but can you set a strategy that always does a "no-full" or "only-incremental" AND can also be forced with amadmin to do full dumps on demand? If yes, perhaps you can use just a single configuration. The incremental only strategy would only cause incrementals daily except the one following your forced "amadmin full" Then dump that "file tape" to real tape. -- Jon H. LaBadie [EMAIL PROTECTED] JG Computing 4455 Province Line Road(609) 252-0159 Princeton, NJ 08540-4322 (609) 683-7220 (fax)
Re: Backup plan needs advice.
Hi Martin I forgot to state that this is a tapeless setup to back up to disk. Then the full backup will be transferred to tape, and also is kept on disk for one week. Martin Hepworth wrote: Bao Amanda will need two separate config's to do this, BUT they will have idea of each others existance so the 'daily' will to full backups as and when the amanda planner thinks it's time to do one. I am _not_ letting Amanda schedule as to when to do a full backup, as is recommended. What I want (actually, my boss) is to force it to run a full every Monday, then a level 1 every other day for the rest of the week, according to the most recent Monday. I hope the above helps clarify it. Sorry for the confusion.
Re: Backup plan needs advice.
Bao Amanda will need two separate config's to do this, BUT they will have idea of each others existance so the 'daily' will to full backups as and when the amanda planner thinks it's time to do one. -- Martin Hepworth Senior Systems Administrator Solid State Logic Ltd +44 (0)1865 842300 bao wrote: Hello, After a very long and hard time struggling with Amanda, I figured out that it's the Athlon bug, and have switched to an Intel. Now it works. However, I am still confused of some things regarding the backup plan, and hope someone will help me answer them. I would like to back up everyday, in incremental, and a weekly full backup once a week, on a specified day. Let the full be on Monday, and the incrementals be on the rest of the week. I now have to configs: 1. Weekly: has dumpcycle=0, runspercycle=1, tapecycle=1, strategy=noinc, index=yes, and record=no 2. Daily: has dumpcycle=7, runspercycle=6, tapecycle=6, strategy=nofull, , index=yes, record=yes The Weekly full will call amadmin to set to run level 0 before it runs. Q1: If I want the incrementals of a week to follow from that week's full, how do I set the record?? (Mon : level 0, Tues-Sun : level 1 - backing up any changes from the most recent Monday) Q2. The Weekly full backup will be transferred to tape with a cycle of 8 tapes, then rotate. What do I need to copy to tape beside the backup image?? index ?? log ?? debug files ?? Thank you all, ** This email and any files transmitted with it are confidential and intended solely for the use of the individual or entity to whom they are addressed. If you have received this email in error please notify the system manager. This footnote also confirms that this email message has been swept by MIMEsweeper for the presence of computer viruses. www.mimesweeper.com **
Backup plan needs advice.
Hello, After a very long and hard time struggling with Amanda, I figured out that it's the Athlon bug, and have switched to an Intel. Now it works. However, I am still confused of some things regarding the backup plan, and hope someone will help me answer them. I would like to back up everyday, in incremental, and a weekly full backup once a week, on a specified day. Let the full be on Monday, and the incrementals be on the rest of the week. I now have to configs: 1. Weekly: has dumpcycle=0, runspercycle=1, tapecycle=1, strategy=noinc, index=yes, and record=no 2. Daily: has dumpcycle=7, runspercycle=6, tapecycle=6, strategy=nofull, , index=yes, record=yes The Weekly full will call amadmin to set to run level 0 before it runs. Q1: If I want the incrementals of a week to follow from that week's full, how do I set the record?? (Mon : level 0, Tues-Sun : level 1 - backing up any changes from the most recent Monday) Q2. The Weekly full backup will be transferred to tape with a cycle of 8 tapes, then rotate. What do I need to copy to tape beside the backup image?? index ?? log ?? debug files ?? Thank you all,
Re: backup plan
On Mon, 31 Mar 2003, Lucia Mazzoni wrote: > Hi everyone, > I need a little help to tune a backup plan with Amanda. > In my company we were used, before Amanda, to have this plan for our > main servers (each one with a dedicated tape device): > - a full backup was made every day from monday to friday but > 1) from monday to thursday tapes were always the same, that is, every > week they were overwritten > 2) on friday we had 4 tapes to save a full-backup per week for one > month. After 4 weeks we selected the oldest tape, put it in a safe place > (against fire) and took a new one for the backup job. > We kept 1 tape per month for 6 months and then one every 3 or 6 months. > > Now I'd like to maintain the same plan with Amanda and my problem is > with Friday tapes that I want to "extract" and put away. > > I put a tapecycle of 4 tapes on Friday configuration but: > - How can I maintain the possibility of restore data from the extracted > tapes? Changing labels every time? Did i get the plan correctly - after a month the oldest Friday dump of one tape goest to archive and a new one replaces it? if this is the case, i think, that you could have the Friday's set to acomodate a lot of numbers (e.g. Friday[0-9][0-9][0-9] for 10-years of archive - 120 tapes total, i.e. Friday001, Friday076, etc), so that amanda has enough name-space. after a month when the archved tape is selected, run command: amadmin no-reuse which tell's amanda not to reuse this tape anymore and to expect a fresh tape when this tape's turn comes (i hope that amanda labels the new tape on the fly, but i wouln'd be sure). This should keep the data located on the tape still indexed in database and therefore restorable via amrecover. another way is to just replace the old tape with new one by simply amlabeling new one with -f option. Note that this way you no longer can amrecover from this tape, but have to amrestore instead (this means that you have to extract whole image from the tape and restore it and only then you can pick the files you want). Laas Toom
Re: backup plan
On Mon, Mar 31, 2003 at 05:31:02PM +0200, Lucia Mazzoni wrote: > Hi everyone, > I need a little help to tune a backup plan with Amanda. > In my company we were used, before Amanda, to have this plan for our > main servers (each one with a dedicated tape device): > - a full backup was made every day from monday to friday but > 1) from monday to thursday tapes were always the same, that is, every > week they were overwritten > 2) on friday we had 4 tapes to save a full-backup per week for one > month. After 4 weeks we selected the oldest tape, put it in a safe place > (against fire) and took a new one for the backup job. > We kept 1 tape per month for 6 months and then one every 3 or 6 months. > > Now I'd like to maintain the same plan with Amanda and my problem is > with Friday tapes that I want to "extract" and put away. > > I put a tapecycle of 4 tapes on Friday configuration but: > - How can I maintain the possibility of restore data from the extracted > tapes? Changing labels every time? > I'm guessing some things about your old setup and if wrong forget my idea. Guesses: The systems in question do not have tape changers. It was operator responsibility to see that the correct tape was in the drive. If the wrong tape was inserted, the previous backup system would just overwrite it. If that is the case, how about setting tapecycle, dumpcycle, and runspercycle to one. Amanda will always take a new tape whose label meets the pattern and with those settings will reuse any previous tape -- I think. -- Jon H. LaBadie [EMAIL PROTECTED] JG Computing 4455 Province Line Road(609) 252-0159 Princeton, NJ 08540-4322 (609) 683-7220 (fax)
backup plan
Hi everyone, I need a little help to tune a backup plan with Amanda. In my company we were used, before Amanda, to have this plan for our main servers (each one with a dedicated tape device): - a full backup was made every day from monday to friday but 1) from monday to thursday tapes were always the same, that is, every week they were overwritten 2) on friday we had 4 tapes to save a full-backup per week for one month. After 4 weeks we selected the oldest tape, put it in a safe place (against fire) and took a new one for the backup job. We kept 1 tape per month for 6 months and then one every 3 or 6 months. Now I'd like to maintain the same plan with Amanda and my problem is with Friday tapes that I want to "extract" and put away. I put a tapecycle of 4 tapes on Friday configuration but: - How can I maintain the possibility of restore data from the extracted tapes? Changing labels every time? Thank you very much in advance for your help. greetings Lucia -- Lucia Mazzoni ASTER Scienza Tecnologia Impresa - S.Cons.p.a. Area di Ricerca di Bologna Via Gobetti 101 - I-40129 Bologna Tel +39 051 6398099 Fax +39 051 6398131 http://www.aster.it
backup plan needs comments
Hi everyone, I have this backup plan in mind as below, and would hope if anyone can suggest what can be wrong, can be hard to set up, or any suggestions. Anything is appreciated. Second HD (Amanda) Tapes -- Full | IncrementalWeekly(20GB) | Daily(12GB) -- Full - - - - - - - - - - copied to - -> Weekly 1 Day 1 -| Day 2 -| Day 3 -| Day 4 -| Day 5 -| Day 6 -| |- - - copied to - - - - - - - - - -> Daily 1 /--We clean up the 2nd HD at the end of this---/ Full - - - - - - - - - - copied to - -> Weekly 2 Day 1 -| Day 2 -| Day 3 -| Day 4 -| Day 5 -| Day 6 -| |- - - copied to - - - - - - - - - -> Daily 2 /--We clean up the 2nd HD at the end of this---/ We copy the Full on 2nd hard drive to Weekly tape after it's done. At the end of Day 6, we can copy everything onto Daily tape. After 8 weeks, we recycle the tapes. Assumptions: 1. Tapeless backup onto a 2nd hard drive is done first, then is copied to tapes. 2. Tapes have sufficient sizes, so only one tape is needed at a time. 3. And GNU-tar is used with Amanda. Problems with restoring: 1. For restore from Weekly tapes, the whole tape has to be dumped to the hard drive, then amanda is used to select the correct files/folders. 2. For restore from Daily tapes, again, the tape contents have to be dumped to hard drive, then amanda is used. 3. How should Amanda's database/index be handled??? Should they be backed up to tapes along with its data?? 4. If the database/index are to be backed up to tapes, Does it mean when restore, I need to put them back to their correct place, i.e., /tmp/amanda , etc. ??? thank you for any comments.
Re: Specifig backup plan - HELP!
>After looking around for a good OpenSourse backup solution, I >decided to make use of Amanda. Now I'am facing a problem configuring >this software the way it fits my company backup policy. Thus I am asking >you, those who are surely more familiar with this topic. > Here are requirements I need to meet: Then amanda may not be the tool for you. You cannot choose when full dumps are done You cannot easily mix-up monthly and weekly full dumps You need one tape per DAY for incremental back-ups All this can be worked around, but that will need dirty tricks, and you will loose some benefits of Amanda when you want to restore (applying the same tricks again). Even using tapes A1 to A7 one week and tapes B1 to B7 on the other week, is not straight forward. Good luck Olivier
Specifig backup plan - HELP!
After looking around for a good OpenSourse backup solution, I decided to make use of Amanda. Now I'am facing a problem configuring this software the way it fits my company backup policy. Thus I am asking you, those who are surely more familiar with this topic. Here are requirements I need to meet: Tapes: FM1, FM2, FM3 -monthly tapes for full backups with 3 months cycle, so that we have last three full monthly backups. Backups performed on 1st Monday of the month. FW2, FW3, FW4, FW5 - weekly tapes for full backups starting from 2nd week of the month and recycling after 2 or 3 weeks (depending on how many weeks we have in a particular month) . Performed every Monday starting from 2nd week of the month. IW1, IW2 - daily tapes for inremental backups rotating every week. Performed every day. To make everything a bit more understandable, let me give you an example. Lets assume we have a period of 4 months, one of which has 5 weeks. Here is a tapes usage schema for those months (M stands for month, W for week and D - sure you now,:-)): M1 M2 M3M4 - W1 - W2 - W3 - W4 || W1 - W2 - W3 - W4 - W5 || W1 - W2 - W3 - W4 | W1 - W2 - W3 - W4 FM1 FW2FW3 FW4FM2 FW2 FW3FW4 FW5FM3FW2 FW3FW4FM1 FW2 FW3 FW4 IW1 IW2 IW1IW2 IW1 IW2IW1 IW2 . and so on. I hope someone would help me to map this scenario into Amanda's configuration schema(s). Please try to respond ASAP. Thanx and best regards, Mariusz Brylant. ..oo00##~*°°*~oo. ..oo00## Mariusz Brylant ~oo. ..oo00## Sys'n'Net Administrator ~ooo. ..oo00## eCard S.A ~oo. ..oo00##~*OFFICE **~ooo ..oo00## address:Domaniewska 41 ~ooo.. ..oo00## MARD D ~. ..oo00## 02-672 Warsaw ~o ..oo00## Poland ~.. ..oo00## mailto: [EMAIL PROTECTED] ~ooo. ..oo00## www:www.ecard.com.pl ~oo. ..oo00## phone: +48(22)8744490 ~o. ..oo00## fax:+48(22)8744491 ~oo... ..oo00##~*PRIVATE **~oo ..oo00## mailto: [EMAIL PROTECTED] ~ooo ..oo00## mailto: [EMAIL PROTECTED] ~ ..ooOO##~*°°*~..
