Re: restore not finding all incremental levels (SOLVED)
-In response to your message- --received from Jon LaBadie-- On Thu, Apr 19, 2007 at 06:50:44PM -0700, Paul Yeatman wrote: -In response to your message- --received from Jean-Louis Martineau-- Paul Yeatman wrote: So, as I continue to look at the problem, I haven't seen before that an amflush is require for the directories on holding disk to be included in a recovery yet . . . this kinda of seems to be the case now (that is as much a question as it is a statement). I see in the amindexd.###.debug log that it is trying to gzip -dc ... files in the amanda index directory for the disk being restored yet these files don't exist. For instance, the log says this is being executed /bin/gzip -dc '/etc/amanda/cass/index/cass246/_/20070331_6.gz' 2/dev/null | sort '/etc/amanda/cass/index/cass246/_/20070331_6' yet /etc/amanda/cass/index/cass246/_/20070331_6.gz does not exist; It is instead on the holding disk. Does it not exist because I have never run amflush? I swear I've performed tens of recoveries that included files on the holding disk that were (obviously) never amflushed. I'm so confused! amanda will never remove the compressed index files (.gz) while the dump are still on tape or holding disk. You said that you removed some dump from holding disk to debug your previous problem, if you ran amdump while dump was missing, amanda has removed the index file. Ah, I get it! I have another Amanda server to check with and do indeed see that all backups on the holding disk have an associated .gz file in the index directory. I CLEARLY need to change this yet, until now on this server, I have been filling a holding disk, leaving those backups there, and then changing my amanda.conf to point to a new [blank] holding disk. I always figured that in the event of a needed restore, I could change my amanda.conf to point to ALL holding disks. There was a reason I was doing this that I won't bore everyone with now. So now I see that, when changing to a new (empty) holding disk, all my previous, compressed, index files where flushed upon the next amdump run! Yikes!!! This is bad!! The good news is that the data is still there!!! But, boy, sure takes away the power of amrecover! I had no idea the consequence of this! I hope you have a backup of your index files. I suppose in a way I do but, given the current state of things, it seems it would be impossible to figure out what needs to be restored to correctly repopulate my index directory. I will hope for the time being that no restores will be needed until I create stable holding disks and repeat full backups of everything. In the meantime, should a restore be necessary, I will resort to an 'amadmin ... info' of the disk in conjuction with sequential hand-run amrestore's. Shouldn't be too major a job as you are only talking about restoring a single directory tree, the index directory tree. As the dumps are on disk, once you figure an an appropriate command line, you may even be able to automate it with a small shell script/loop. Be aware that you can specify multiple holding disks in your config. And as one of the holding disk parameters is use, you could even specify an appropriate use value for your old holding disk that makes it seem full. Then no more dumps will be added there but the old dumps will still be available for recovery. Thanks again for this! I am aware I can specify multiple holding disks and will do this from now on. I'm surprised that it has been as difficult as it has to make the disks appear full to AMANDA, however. I am using directories on the same RAID disk as holding disks and thus first tried specifying a negative Gigabyte value that was larger than the amount of space available on the RAID disk, for example I specified use -250 Gb when there is roughly 240 Gb available. Strangely, the next backup still used all directories on this disk. As my understanding of specifying use 0 Mb means use all space available, the best I've been able to come with is use 1 Mb which seems to be the smallest amount of space I can specify. This of course means, not very much, but some data is undesirably written into all holding disks (directories) every backup. I feel like I'm not quite understanding how this is supposed to work. When trying various use values, for instance, I don't quite understand what amcheck is trying to tell me. If I set use -250 Gb on one of the holding disks, for instance, amcheck tells me WARNING: holding disk /backup/vtc1/cass: only 238115672 KB free (4032823296 KB requested) The roughly 240 G free is correct but 4 Tb requested??? How does this relate to use -250 Gb? Don't get it! Paul -- Jon H. LaBadie [EMAIL PROTECTED] JG Computing 4455 Province Line Road(609) 252-0159 Princeton, NJ
Re: restore not finding all incremental levels
-In response to your message- --received from Jon LaBadie-- On Tue, Apr 17, 2007 at 07:13:09PM -0700, Paul Yeatman wrote: Hi, I'm testing my backup system and am finding something unexpected. I'm using amanda 2.4.4p3-1 on a server running RedHat linux ES 4. The restore I'm attempting is performed all on the server using a spare disk on the server. When I run amrecover and add--add .--a partition that has 7 levels of incrementals and then type list, it shows the tape (actually the server is a tapeless server so it is really on disk) with the original full backup on it and then levels 6 and 7 on the holding disk. All other levels are skipped. Levels 1-5 are not included in the planned restore. I would expect it to restore the latest of each incremental level 1-7. I currently have several holding disks and can confirm that the incremental directories that an 'amadmin info' shows for the same partition exist. When amrecover is executed, it shows found amanda directory from all expected holding disks and holding disk directories and the directories needed for this restore in particular. If I remove the holding disk from amanda.conf that holds levels 6 and 7, then no levels after the original tape are listed in the planned restore. Same initial thought as JLM, perhaps nothing changed in the directory during the incremental levels 1-5. This particular partition was backed up in 2005-05-04. On the first holding disk, the latest level 1 is in inc directory 20050509, the latest level 2 is in 20060510, and then the rest of the incremental directories on the holding disk (until the next holding disk) are level 3's for this partition. If I include only this one holding disk in my amanda.conf, this is what the amindexd.20070417184329.debug shows (full file attached) I would presume that this is a fairly stable, unchanging storage. I base that guess on the fact that the last full backup was May 2005 and the first incremental a year later in May of 2006. Another clue is that the incrementals haven't been moved from the holding disk to tape (physical or virtual) in the last year. So, as I continue to look at the problem, I haven't seen before that an amflush is require for the directories on holding disk to be included in a recovery yet . . . this kinda of seems to be the case now (that is as much a question as it is a statement). I see in the amindexd.###.debug log that it is trying to gzip -dc ... files in the amanda index directory for the disk being restored yet these files don't exist. For instance, the log says this is being executed /bin/gzip -dc '/etc/amanda/cass/index/cass246/_/20070331_6.gz' 2/dev/null | sort '/etc/amanda/cass/index/cass246/_/20070331_6' yet /etc/amanda/cass/index/cass246/_/20070331_6.gz does not exist; It is instead on the holding disk. Does it not exist because I have never run amflush? I swear I've performed tens of recoveries that included files on the holding disk that were (obviously) never amflushed. I'm so confused! Paul -- Jon H. LaBadie [EMAIL PROTECTED] JG Computing 4455 Province Line Road(609) 252-0159 Princeton, NJ 08540-4322 (609) 683-7220 (fax) -- Paul Yeatman (858) 534-9896[EMAIL PROTECTED] == ==Proudly brought to you by Mutt== ==
Re: restore not finding all incremental levels
On Thu, Apr 19, 2007 at 03:31:33PM -0700, Paul Yeatman wrote: -In response to your message- --received from Jon LaBadie-- On Tue, Apr 17, 2007 at 07:13:09PM -0700, Paul Yeatman wrote: Hi, I'm testing my backup system and am finding something unexpected. ... I would presume that this is a fairly stable, unchanging storage. I base that guess on the fact that the last full backup was May 2005 and the first incremental a year later in May of 2006. Another clue is that the incrementals haven't been moved from the holding disk to tape (physical or virtual) in the last year. So, as I continue to look at the problem, I haven't seen before that an amflush is require for the directories on holding disk to be included in a recovery yet . . . this kinda of seems to be the case now (that is as much a question as it is a statement). I see in the amindexd.###.debug log that it is trying to gzip -dc ... files in the amanda index directory for the disk being restored yet these files don't exist. For instance, the log says this is being executed /bin/gzip -dc '/etc/amanda/cass/index/cass246/_/20070331_6.gz' 2/dev/null | sort '/etc/amanda/cass/index/cass246/_/20070331_6' yet /etc/amanda/cass/index/cass246/_/20070331_6.gz does not exist; It is instead on the holding disk. This surprises me, are you sure your index files are on the holding disk? I.e. you can ls /etc/amanda/cass/index/cass246/_/20070331_6.gz and it is not there, but it is on the holding disk? What is its holding disk path? I may have missed it, but I didn't think index files ever end up on the holding disk. -- Jon H. LaBadie [EMAIL PROTECTED] JG Computing 4455 Province Line Road(609) 252-0159 Princeton, NJ 08540-4322 (609) 683-7220 (fax)
Re: restore not finding all incremental levels
Paul Yeatman wrote: So, as I continue to look at the problem, I haven't seen before that an amflush is require for the directories on holding disk to be included in a recovery yet . . . this kinda of seems to be the case now (that is as much a question as it is a statement). I see in the amindexd.###.debug log that it is trying to gzip -dc ... files in the amanda index directory for the disk being restored yet these files don't exist. For instance, the log says this is being executed /bin/gzip -dc '/etc/amanda/cass/index/cass246/_/20070331_6.gz' 2/dev/null | sort '/etc/amanda/cass/index/cass246/_/20070331_6' yet /etc/amanda/cass/index/cass246/_/20070331_6.gz does not exist; It is instead on the holding disk. Does it not exist because I have never run amflush? I swear I've performed tens of recoveries that included files on the holding disk that were (obviously) never amflushed. I'm so confused! amanda will never remove the compressed index files (.gz) while the dump are still on tape or holding disk. You said that you removed some dump from holding disk to debug your previous problem, if you ran amdump while dump was missing, amanda has removed the index file. I hope you have a backup of your index files. Jean-Louis
Re: restore not finding all incremental levels
-In response to your message- --received from Jon LaBadie-- On Thu, Apr 19, 2007 at 03:31:33PM -0700, Paul Yeatman wrote: -In response to your message- --received from Jon LaBadie-- On Tue, Apr 17, 2007 at 07:13:09PM -0700, Paul Yeatman wrote: Hi, I'm testing my backup system and am finding something unexpected. ... I would presume that this is a fairly stable, unchanging storage. I base that guess on the fact that the last full backup was May 2005 and the first incremental a year later in May of 2006. Another clue is that the incrementals haven't been moved from the holding disk to tape (physical or virtual) in the last year. So, as I continue to look at the problem, I haven't seen before that an amflush is require for the directories on holding disk to be included in a recovery yet . . . this kinda of seems to be the case now (that is as much a question as it is a statement). I see in the amindexd.###.debug log that it is trying to gzip -dc ... files in the amanda index directory for the disk being restored yet these files don't exist. For instance, the log says this is being executed /bin/gzip -dc '/etc/amanda/cass/index/cass246/_/20070331_6.gz' 2/dev/null | sort '/etc/amanda/cass/index/cass246/_/20070331_6' yet /etc/amanda/cass/index/cass246/_/20070331_6.gz does not exist; It is instead on the holding disk. This surprises me, are you sure your index files are on the holding disk? I.e. you can ls /etc/amanda/cass/index/cass246/_/20070331_6.gz and it is not there, but it is on the holding disk? What is its holding disk path? I may have missed it, but I didn't think index files ever end up on the holding disk. Ack! Sorry, Jon. Once again I'm not being very clear. What I mean to say is that a file in the index directory, namely /etc/amanda/cass/index/cass246/_/20070331_6.gz, does not exist (as amindexd appears to be looking for) yet the backup IS indeed on the holding disk, ie. /../daily1/22070331/cass246._.6. I guess another way to put this is to say there is not a .gz file in the index directory for any incremental backup on the holding disk. If there should be, then things are really screwed up (and I have no idea how I would have unintentionally deleted them)! I'm thinking that this would only be the case if I've run amflush (which I have never run). The only .gz files in the index directory for this particular disk are the two tape runs I've made. All other backups are still on the holding disk and do not have a representative .gz file in the index directory. Is this not the way it should be? Thanks again to Jon and Jean-Louis for all the help! Paul -- Jon H. LaBadie [EMAIL PROTECTED] JG Computing 4455 Province Line Road(609) 252-0159 Princeton, NJ 08540-4322 (609) 683-7220 (fax) -- Paul Yeatman (858) 534-9896[EMAIL PROTECTED] == ==Proudly brought to you by Mutt== ==
Re: restore not finding all incremental levels (SOLVED)
-In response to your message- --received from Jean-Louis Martineau-- Paul Yeatman wrote: So, as I continue to look at the problem, I haven't seen before that an amflush is require for the directories on holding disk to be included in a recovery yet . . . this kinda of seems to be the case now (that is as much a question as it is a statement). I see in the amindexd.###.debug log that it is trying to gzip -dc ... files in the amanda index directory for the disk being restored yet these files don't exist. For instance, the log says this is being executed /bin/gzip -dc '/etc/amanda/cass/index/cass246/_/20070331_6.gz' 2/dev/null | sort '/etc/amanda/cass/index/cass246/_/20070331_6' yet /etc/amanda/cass/index/cass246/_/20070331_6.gz does not exist; It is instead on the holding disk. Does it not exist because I have never run amflush? I swear I've performed tens of recoveries that included files on the holding disk that were (obviously) never amflushed. I'm so confused! amanda will never remove the compressed index files (.gz) while the dump are still on tape or holding disk. You said that you removed some dump from holding disk to debug your previous problem, if you ran amdump while dump was missing, amanda has removed the index file. Ah, I get it! I have another Amanda server to check with and do indeed see that all backups on the holding disk have an associated .gz file in the index directory. I CLEARLY need to change this yet, until now on this server, I have been filling a holding disk, leaving those backups there, and then changing my amanda.conf to point to a new [blank] holding disk. I always figured that in the event of a needed restore, I could change my amanda.conf to point to ALL holding disks. There was a reason I was doing this that I won't bore everyone with now. So now I see that, when changing to a new (empty) holding disk, all my previous, compressed, index files where flushed upon the next amdump run! Yikes!!! This is bad!! The good news is that the data is still there!!! But, boy, sure takes away the power of amrecover! I had no idea the consequence of this! I hope you have a backup of your index files. I suppose in a way I do but, given the current state of things, it seems it would be impossible to figure out what needs to be restored to correctly repopulate my index directory. I will hope for the time being that no restores will be needed until I create stable holding disks and repeat full backups of everything. In the meantime, should a restore be necessary, I will resort to an 'amadmin ... info' of the disk in conjuction with sequential hand-run amrestore's. Thanks Jon and Jean-Louis! Paul Jean-Louis -- Paul Yeatman (858) 534-9896[EMAIL PROTECTED] == ==Proudly brought to you by Mutt== ==
Re: restore not finding all incremental levels (SOLVED)
On Thu, Apr 19, 2007 at 06:50:44PM -0700, Paul Yeatman wrote: -In response to your message- --received from Jean-Louis Martineau-- Paul Yeatman wrote: So, as I continue to look at the problem, I haven't seen before that an amflush is require for the directories on holding disk to be included in a recovery yet . . . this kinda of seems to be the case now (that is as much a question as it is a statement). I see in the amindexd.###.debug log that it is trying to gzip -dc ... files in the amanda index directory for the disk being restored yet these files don't exist. For instance, the log says this is being executed /bin/gzip -dc '/etc/amanda/cass/index/cass246/_/20070331_6.gz' 2/dev/null | sort '/etc/amanda/cass/index/cass246/_/20070331_6' yet /etc/amanda/cass/index/cass246/_/20070331_6.gz does not exist; It is instead on the holding disk. Does it not exist because I have never run amflush? I swear I've performed tens of recoveries that included files on the holding disk that were (obviously) never amflushed. I'm so confused! amanda will never remove the compressed index files (.gz) while the dump are still on tape or holding disk. You said that you removed some dump from holding disk to debug your previous problem, if you ran amdump while dump was missing, amanda has removed the index file. Ah, I get it! I have another Amanda server to check with and do indeed see that all backups on the holding disk have an associated .gz file in the index directory. I CLEARLY need to change this yet, until now on this server, I have been filling a holding disk, leaving those backups there, and then changing my amanda.conf to point to a new [blank] holding disk. I always figured that in the event of a needed restore, I could change my amanda.conf to point to ALL holding disks. There was a reason I was doing this that I won't bore everyone with now. So now I see that, when changing to a new (empty) holding disk, all my previous, compressed, index files where flushed upon the next amdump run! Yikes!!! This is bad!! The good news is that the data is still there!!! But, boy, sure takes away the power of amrecover! I had no idea the consequence of this! I hope you have a backup of your index files. I suppose in a way I do but, given the current state of things, it seems it would be impossible to figure out what needs to be restored to correctly repopulate my index directory. I will hope for the time being that no restores will be needed until I create stable holding disks and repeat full backups of everything. In the meantime, should a restore be necessary, I will resort to an 'amadmin ... info' of the disk in conjuction with sequential hand-run amrestore's. Shouldn't be too major a job as you are only talking about restoring a single directory tree, the index directory tree. As the dumps are on disk, once you figure an an appropriate command line, you may even be able to automate it with a small shell script/loop. Be aware that you can specify multiple holding disks in your config. And as one of the holding disk parameters is use, you could even specify an appropriate use value for your old holding disk that makes it seem full. Then no more dumps will be added there but the old dumps will still be available for recovery. -- Jon H. LaBadie [EMAIL PROTECTED] JG Computing 4455 Province Line Road(609) 252-0159 Princeton, NJ 08540-4322 (609) 683-7220 (fax)
Re: restore not finding all incremental levels
On Tue, Apr 17, 2007 at 07:13:09PM -0700, Paul Yeatman wrote: Hi, I'm testing my backup system and am finding something unexpected. I'm using amanda 2.4.4p3-1 on a server running RedHat linux ES 4. The restore I'm attempting is performed all on the server using a spare disk on the server. When I run amrecover and add--add .--a partition that has 7 levels of incrementals and then type list, it shows the tape (actually the server is a tapeless server so it is really on disk) with the original full backup on it and then levels 6 and 7 on the holding disk. All other levels are skipped. Levels 1-5 are not included in the planned restore. I would expect it to restore the latest of each incremental level 1-7. I currently have several holding disks and can confirm that the incremental directories that an 'amadmin info' shows for the same partition exist. When amrecover is executed, it shows found amanda directory from all expected holding disks and holding disk directories and the directories needed for this restore in particular. If I remove the holding disk from amanda.conf that holds levels 6 and 7, then no levels after the original tape are listed in the planned restore. Same initial thought as JLM, perhaps nothing changed in the directory during the incremental levels 1-5. This particular partition was backed up in 2005-05-04. On the first holding disk, the latest level 1 is in inc directory 20050509, the latest level 2 is in 20060510, and then the rest of the incremental directories on the holding disk (until the next holding disk) are level 3's for this partition. If I include only this one holding disk in my amanda.conf, this is what the amindexd.20070417184329.debug shows (full file attached) I would presume that this is a fairly stable, unchanging storage. I base that guess on the fact that the last full backup was May 2005 and the first incremental a year later in May of 2006. Another clue is that the incrementals haven't been moved from the holding disk to tape (physical or virtual) in the last year. -- Jon H. LaBadie [EMAIL PROTECTED] JG Computing 4455 Province Line Road(609) 252-0159 Princeton, NJ 08540-4322 (609) 683-7220 (fax)
Re: restore not finding all incremental levels
-In response to your message- --received from Jean-Louis Martineau-- Maybe all file included in level 1-5 are also included in level 6-7, level 1-5 are not needed for recovery. You can find it by running zdiff on your index files. Jean-Louis Thanks, Jean-Louis. I thought about this but I have never seen this type of behavior before after many years of running Amanda. I've always seen Amanda methodically go through all of the incremental levels even if all a level contained were directories (changed dates, likely). This is a much newer version of Amanda than I've used most of the time, however, and is possibly newer and smarter Amanda behavior...? Paul Paul Yeatman wrote: Hi, I'm testing my backup system and am finding something unexpected. I'm using amanda 2.4.4p3-1 on a server running RedHat linux ES 4. The restore I'm attempting is performed all on the server using a spare disk on the server. When I run amrecover and add--add .--a partition that has 7 levels of incrementals and then type list, it shows the tape (actually the server is a tapeless server so it is really on disk) with the original full backup on it and then levels 6 and 7 on the holding disk. All other levels are skipped. Levels 1-5 are not included in the planned restore. I would expect it to restore the latest of each incremental level 1-7. I currently have several holding disks and can confirm that the incremental directories that an 'amadmin info' shows for the same partition exist. When amrecover is executed, it shows found amanda directory from all expected holding disks and holding disk directories and the directories needed for this restore in particular. If I remove the holding disk from amanda.conf that holds levels 6 and 7, then no levels after the original tape are listed in the planned restore. This particular partition was backed up in 2005-05-04. On the first holding disk, the latest level 1 is in inc directory 20050509, the latest level 2 is in 20060510, and then the rest of the incremental directories on the holding disk (until the next holding disk) are level 3's for this partition. If I include only this one holding disk in my amanda.conf, this is what the amindexd.20070417184329.debug shows (full file attached) amindexd: time 69.178: removing index file: /etc/amanda/cass/index/cass246/_/200 60504_0 amindexd: time 69.182: removing index file: /etc/amanda/cass/index/cass246/_/200 60509_1 amindexd: time 69.182: removing index file: /etc/amanda/cass/index/cass246/_/200 60510_2 amindexd: time 69.182: removing index file: /etc/amanda/cass/index/cass246/_/200 60525_3 amindexd: time 69.183: 200 Good bye. amindexd: time 69.183: pid 29011 finish time Tue Apr 17 18:44:39 2007 Why is it removing index file: blah? To save space, uncompressed index file are not needed. Thanks, Paul -- Paul Yeatman (858) 534-9896[EMAIL PROTECTED] == ==Proudly brought to you by Mutt== ==
Re: restore not finding all incremental levels
-In response to your message- --received from Jon LaBadie-- On Tue, Apr 17, 2007 at 07:13:09PM -0700, Paul Yeatman wrote: Hi, I'm testing my backup system and am finding something unexpected. I'm using amanda 2.4.4p3-1 on a server running RedHat linux ES 4. The restore I'm attempting is performed all on the server using a spare disk on the server. When I run amrecover and add--add .--a partition that has 7 levels of incrementals and then type list, it shows the tape (actually the server is a tapeless server so it is really on disk) with the original full backup on it and then levels 6 and 7 on the holding disk. All other levels are skipped. Levels 1-5 are not included in the planned restore. I would expect it to restore the latest of each incremental level 1-7. I currently have several holding disks and can confirm that the incremental directories that an 'amadmin info' shows for the same partition exist. When amrecover is executed, it shows found amanda directory from all expected holding disks and holding disk directories and the directories needed for this restore in particular. If I remove the holding disk from amanda.conf that holds levels 6 and 7, then no levels after the original tape are listed in the planned restore. Same initial thought as JLM, perhaps nothing changed in the directory during the incremental levels 1-5. I have never seen Amanda do this before yet possibly new behavior. I mentioned more in my response to JLM. This particular partition was backed up in 2005-05-04. On the first holding disk, the latest level 1 is in inc directory 20050509, the latest level 2 is in 20060510, and then the rest of the incremental directories on the holding disk (until the next holding disk) are level 3's for this partition. If I include only this one holding disk in my amanda.conf, this is what the amindexd.20070417184329.debug shows (full file attached) I would presume that this is a fairly stable, unchanging storage. I base that guess on the fact that the last full backup was May 2005 and the first incremental a year later in May of 2006. Another clue is that the incrementals haven't been moved from the holding disk to tape (physical or virtual) in the last year. Very sorry about that. The 2005-05-04 should have been 2006-05-04, so the bump from level 1 to level 2 occurred five days later on 20060510. So, yes, it has not gone through all 9 levels in a year's time but not quite as stable as my error made it appear. Also, as for the clue, I have an unusual config for which I am making full backups once a year leaving everything on the holding disk in between (never running amflush and keeping dual copies of both tapes and holding disk). I'm right at the point of making another set of full backups and so am taking the time to make sure the set I have is fully recoverable. Paul -- Jon H. LaBadie [EMAIL PROTECTED] JG Computing 4455 Province Line Road(609) 252-0159 Princeton, NJ 08540-4322 (609) 683-7220 (fax) -- Paul Yeatman (858) 534-9896[EMAIL PROTECTED] == ==Proudly brought to you by Mutt== ==
restore not finding all incremental levels
Hi, I'm testing my backup system and am finding something unexpected. I'm using amanda 2.4.4p3-1 on a server running RedHat linux ES 4. The restore I'm attempting is performed all on the server using a spare disk on the server. When I run amrecover and add--add .--a partition that has 7 levels of incrementals and then type list, it shows the tape (actually the server is a tapeless server so it is really on disk) with the original full backup on it and then levels 6 and 7 on the holding disk. All other levels are skipped. Levels 1-5 are not included in the planned restore. I would expect it to restore the latest of each incremental level 1-7. I currently have several holding disks and can confirm that the incremental directories that an 'amadmin info' shows for the same partition exist. When amrecover is executed, it shows found amanda directory from all expected holding disks and holding disk directories and the directories needed for this restore in particular. If I remove the holding disk from amanda.conf that holds levels 6 and 7, then no levels after the original tape are listed in the planned restore. This particular partition was backed up in 2005-05-04. On the first holding disk, the latest level 1 is in inc directory 20050509, the latest level 2 is in 20060510, and then the rest of the incremental directories on the holding disk (until the next holding disk) are level 3's for this partition. If I include only this one holding disk in my amanda.conf, this is what the amindexd.20070417184329.debug shows (full file attached) amindexd: time 69.178: removing index file: /etc/amanda/cass/index/cass246/_/200 60504_0 amindexd: time 69.182: removing index file: /etc/amanda/cass/index/cass246/_/200 60509_1 amindexd: time 69.182: removing index file: /etc/amanda/cass/index/cass246/_/200 60510_2 amindexd: time 69.182: removing index file: /etc/amanda/cass/index/cass246/_/200 60525_3 amindexd: time 69.183: 200 Good bye. amindexd: time 69.183: pid 29011 finish time Tue Apr 17 18:44:39 2007 Why is it removing index file: blah? Thanks, Paul -- Paul Yeatman (858) 534-9896[EMAIL PROTECTED] == ==Proudly brought to you by Mutt== == amindexd: debug 1 pid 29011 ruid 33 euid 33: start at Tue Apr 17 18:43:29 2007 amindexd: version 2.4.4p3 amindexd: time 0.000: 220 cass251 AMANDA index server (2.4.4p3) ready. amindexd: time 0.000: SECURITY USER root amindexd: time 0.000: bsd security: remote host localhost.localdomain user root local user amanda amindexd: time 0.001: amandahosts security check passed amindexd: time 0.001: 200 Access OK amindexd: time 0.040: FEATURES feff9ffe0f amindexd: time 0.040: 200 FEATURES feff9ffe0f amindexd: time 0.080: DATE 2007-04-17 amindexd: time 0.080: 200 Working date set to 2007-04-17. amindexd: time 0.120: SCNF cass amindexd: time 0.173: 200 Config set to cass. amindexd: time 0.173: HOST cass251.ucsd.edu amindexd: time 0.173: 501 Host cass251.ucsd.edu is not in your disklist. amindexd: time 0.213: HOST cass251.ucsd.edu amindexd: time 0.213: 501 Host cass251.ucsd.edu is not in your disklist. amindexd: time 0.253: HOST cass251 amindexd: time 0.253: 200 Dump host set to cass251. amindexd: time 0.294: DISK /tmp/246_ amindexd: time 0.294: 501 Disk cass251:/tmp/246_ is not in your disklist. amindexd: time 0.334: DISK sda1 amindexd: time 0.334: 501 Disk cass251:sda1 is not in your disklist. amindexd: time 5.420: HOST cass246 amindexd: time 5.420: 200 Dump host set to cass246. amindexd: time 26.662: DATE 2006-05-30 amindexd: time 26.663: 200 Working date set to 2006-05-30. amindexd: time 32.715: DISK / amindexd: time 32.716: - 2006-10-24 6 cass_49 105 amindexd: time 32.716: - 2006-05-25 3 /backup/vtc1/cass/20060525/cass246._.3 0 amindexd: time 32.716: - 2006-05-24 3 /backup/vtc1/cass/20060524/cass246._.3 0 amindexd: time 32.716: - 2006-05-23 3 /backup/vtc1/cass/20060523/cass246._.3 0 amindexd: time 32.716: - 2006-05-20 3 /backup/vtc1/cass/20060520/cass246._.3 0 amindexd: time 32.716: - 2006-05-19 3 /backup/vtc1/cass/20060519/cass246._.3 0 amindexd: time 32.716: - 2006-05-18 3 /backup/vtc1/cass/20060518/cass246._.3 0 amindexd: time 32.716: - 2006-05-17 3 /backup/vtc1/cass/20060517/cass246._.3 0 amindexd: time 32.716: - 2006-05-16 3 /backup/vtc1/cass/20060516/cass246._.3 0 amindexd: time 32.716: - 2006-05-13 3 /backup/vtc1/cass/20060513/cass246._.3 0 amindexd: time 32.716: - 2006-05-11 3 /backup/vtc1/cass/20060511/cass246._.3 0 amindexd: time 32.716: - 2006-05-10 2 /backup/vtc1/cass/20060510/cass246._.2 0 amindexd: time 32.716: - 2006-05-09 1 /backup/vtc1/cass/20060509/cass246._.1 0 amindexd: time 32.716: - 2006-05-06 1 /backup/vtc1/cass/20060506/cass246._.1 0 amindexd: time 32.716: - 2006-05-05 1 /backup/vtc1/cass/20060505/cass246._.1 0 amindexd: time 32.716: - 2006-05-04 0 cass_01 3 amindexd: time 32.716: 200 Disk set
Defining backup levels
What exactly is meant by backup levels 1 2? For instance, when a backup bumps to level 2, does that mean that the last level1 backup has all the changes since level0, and the level 2 backups will have everything since the bump date? BTW, I haven't forgotten about the restoring from DVD - I've been away and it's taking me a bit to catch up. I'll report back when I have managed to spend some time with that. Anne pgp9sg6lcdYOl.pgp Description: PGP signature
Re: Defining backup levels
On Wed, 16 Aug 2006 at 6:17pm, Anne Wilson wrote What exactly is meant by backup levels 1 2? For instance, when a backup bumps to level 2, does that mean that the last level1 backup has all the changes since level0, and the level 2 backups will have everything since the bump date? From 'man dump': -level# The dump level (any integer). A level 0, full backup, guaran- tees the entire file system is copied (but see also the -h option below). A level number above 0, incremental backup, tells dump to copy all files new or modified since the last dump of a lower level. The default level is 9. Historically only levels 0 to 9 were usable in dump, this version is able to understand any integer as a dump level. IOW, a level 2 only grabs stuff that has changed since the last level 1. -- Joshua Baker-LePain Department of Biomedical Engineering Duke University
Re: Defining backup levels
On Wednesday 16 August 2006 18:29, Joshua Baker-LePain wrote: On Wed, 16 Aug 2006 at 6:17pm, Anne Wilson wrote What exactly is meant by backup levels 1 2? For instance, when a backup bumps to level 2, does that mean that the last level1 backup has all the changes since level0, and the level 2 backups will have everything since the bump date? From 'man dump': -level# The dump level (any integer). A level 0, full backup, guaran- tees the entire file system is copied (but see also the -h option below). A level number above 0, incremental backup, tells dump to copy all files new or modified since the last dump of a lower level. The default level is 9. Historically only levels 0 to 9 were usable in dump, this version is able to understand any integer as a dump level. I had read that, but still wasn't entirely sure, probably because I read it when I was still struggling to understand several other things at the same time. IOW, a level 2 only grabs stuff that has changed since the last level 1. That's what I thought. Thanks for the clarification and reassurance Anne pgpKQ6FMQGsqa.pgp Description: PGP signature
Re: Defining backup levels
Joshua Baker-LePain [EMAIL PROTECTED] writes: From 'man dump': -level# The dump level (any integer). A level 0, full backup, guaran- tees the entire file system is copied (but see also the -h option below). A level number above 0, incremental backup, tells dump to copy all files new or modified since the last dump of a lower level. The default level is 9. Historically only levels 0 to 9 were usable in dump, this version is able to understand any integer as a dump level. IOW, a level 2 only grabs stuff that has changed since the last level 1. I think if you do 0 1 0 2 then the level 2 will dump things changed since the second 0. FWIW, the NetBSD man page is similar: -0-9Dump levels. A level 0, full backup, guarantees the entire file system is copied (but see also the -h option below). A level number above 0, incremental backup, tells dump to copy all files new or modified since the last dump of a lower level. The default level is 9. It's amusing that someone wanted to extend past 9. I've only ever seen 3 or maybe 4 in real use. -- Greg Troxel [EMAIL PROTECTED]
Re: Defining backup levels
On Wed, Aug 16, 2006 at 08:23:51PM -0400, Greg Troxel wrote: Joshua Baker-LePain [EMAIL PROTECTED] writes: From 'man dump': -level# The dump level (any integer). A level 0, full backup, guaran- tees the entire file system is copied (but see also the -h option below). A level number above 0, incremental backup, tells dump to copy all files new or modified since the last dump of a lower level. The default level is 9. Historically only levels 0 to 9 were usable in dump, this version is able to understand any integer as a dump level. IOW, a level 2 only grabs stuff that has changed since the last level 1. I think if you do 0 1 0 2 then the level 2 will dump things changed since the second 0. That is true, but ... Amanda doesn't skip levels so it would not happen in amanda. And amanda does not go up levels (except to 0) so the sequence 0 1 2 3 4 2 will not occur either. Though done manually, that last level 2 would be be all since the second dump, a level 1. -- Jon H. LaBadie [EMAIL PROTECTED] JG Computing 4455 Province Line Road(609) 252-0159 Princeton, NJ 08540-4322 (609) 683-7220 (fax)
Re: New question on levels
On 2006-08-02 04:50, Jon LaBadie wrote: On Wed, Aug 02, 2006 at 12:36:57AM +0200, Josef Wolf wrote: On Wed, Jul 26, 2006 at 11:11:41AM -0400, Jon LaBadie wrote: If talking about amanda's planning, it shouldn't happen. I'm pretty sure that amanda does at most, estimates of three dump levels; level 0, the last dump level, and the last level +1. So amanda will never fall back to lower levels once a bump was done? Same question I asked the day after I wrote that. Got no replies. Back from holiday now :-) No, amanda currenlty only increases the backuplevel. And decreases only to 0 again. -- Paul Bijnens, xplanation Technology ServicesTel +32 16 397.511 Technologielaan 21 bus 2, B-3001 Leuven, BELGIUMFax +32 16 397.512 http://www.xplanation.com/ email: [EMAIL PROTECTED] *** * I think I've got the hang of it now: exit, ^D, ^C, ^\, ^Z, ^Q, ^^, * * F6, quit, ZZ, :q, :q!, M-Z, ^X^C, logoff, logout, close, bye, /bye, * * stop, end, F3, ~., ^]c, +++ ATH, disconnect, halt, abort, hangup, * * PF4, F20, ^X^X, :D::D, KJOB, F14-f-e, F8-e, kill -1 $$, shutdown, * * init 0, kill -9 1, Alt-F4, Ctrl-Alt-Del, AltGr-NumLock, Stop-A, ... * * ... Are you sure? ... YES ... Phew ... I'm out * ***
Re: New question on levels
On Wed, Jul 26, 2006 at 11:11:41AM -0400, Jon LaBadie wrote: If talking about amanda's planning, it shouldn't happen. I'm pretty sure that amanda does at most, estimates of three dump levels; level 0, the last dump level, and the last level +1. So amanda will never fall back to lower levels once a bump was done?
Re: New question on levels
On Tuesday 01 August 2006 18:36, Josef Wolf wrote: On Wed, Jul 26, 2006 at 11:11:41AM -0400, Jon LaBadie wrote: If talking about amanda's planning, it shouldn't happen. I'm pretty sure that amanda does at most, estimates of three dump levels; level 0, the last dump level, and the last level +1. So amanda will never fall back to lower levels once a bump was done? Not to my knowledge. Only a level 0 is a reset. -- Cheers, Gene People having trouble with vz bouncing email to me should add the word 'online' between the 'verizon', and the dot which bypasses vz's stupid bounce rules. I do use spamassassin too. :-) Yahoo.com and AOL/TW attorneys please note, additions to the above message by Gene Heskett are: Copyright 2006 by Maurice Eugene Heskett, all rights reserved.
Re: New question on levels
On Wed, Aug 02, 2006 at 12:36:57AM +0200, Josef Wolf wrote: On Wed, Jul 26, 2006 at 11:11:41AM -0400, Jon LaBadie wrote: If talking about amanda's planning, it shouldn't happen. I'm pretty sure that amanda does at most, estimates of three dump levels; level 0, the last dump level, and the last level +1. So amanda will never fall back to lower levels once a bump was done? Same question I asked the day after I wrote that. Got no replies. -- Jon H. LaBadie [EMAIL PROTECTED] JG Computing 4455 Province Line Road(609) 252-0159 Princeton, NJ 08540-4322 (609) 683-7220 (fax)
Re: more fun with levels and estimates fubar
* Jon LaBadie [EMAIL PROTECTED] [20060727 12:11]: On Thu, Jul 27, 2006 at 10:26:28AM -0400, Jean-Francois Malouin wrote: Continuing with my estimate saga... What would cause a level 1 backup of a DLE with gnutar be as big as a full? Estimates are done with calcsize and gnutar is 1.13.25 with amanda-2.4.5 Could it be that the gnutar-lists file is corrupted? Looking at them I see that they all have the same size for level 0 1 and 2...Normal? Are the gnutar-lists files current? I.e. are they being updated each night? My similar problem had the system creating *.NEW files, but never renaming them after the dump. So I was always using old list files. But record no (my problem) is not yours. They seem to be current, but I don't understand how and when they are generated. And what's the content actually. Any way of making sense of it? I recall from your original post that you were having problems with same size dumps from DLEs the used gnutar and others that used xfsdump. The latter would not use gnutar-lists so I doubt the overall problem was there. Correct. This is what is stumping me. xfsdump keeps an inventory and that client had close to ~4000 entries in there. I've checked and pruned them. But just in case, what about permissions/ownership on the gnutar-list directories and their parent paths. What about the curinfo files. Are they being updated nightly? all fine too, permission-wise that is. The solution to my problem was triggered by a several year-old thread were all levels were the same size. The solution to that posters troubles was to stop running a script that nightly touched or chmod'ed all files on the DLEs. Might you be having a similar situation such that all ls -l or ls -cl timestamps are current thus making it look like they require backing up? I'll check into that but a first glance this is not the case. I'll report back when I've sorted out this mess. Thanks for tips John! jf -- Jon H. LaBadie [EMAIL PROTECTED] JG Computing 4455 Province Line Road(609) 252-0159 Princeton, NJ 08540-4322 (609) 683-7220 (fax) -- °
more fun with levels and estimates fubar
Continuing with my estimate saga... What would cause a level 1 backup of a DLE with gnutar be as big as a full? Estimates are done with calcsize and gnutar is 1.13.25 with amanda-2.4.5 Could it be that the gnutar-lists file is corrupted? Looking at them I see that they all have the same size for level 0 1 and 2...Normal? Some debug info: sendsize debug: sendsize[1678914]: estimate size for koenigstiger1_3 level 0: 48848631 KB sendsize[1678914]: estimate size for koenigstiger1_3 level 1: 2343 KB sendsize[1678914]: estimate size for koenigstiger1_3 level 2: 2343 KB sendbackup debug: sendbackup: debug 1 pid 2017899 ruid 666 euid 666: start at Tue Jul 25 19:25:06 2006 /opt/amanda/amanda5/libexec/sendbackup: version 2.4.5 parsed request as: program `GNUTAR' disk `koenigstiger1_3' device `/data/koenigstiger/koenigstiger1/katarina' level 1 since 2006:7:19:16:27:46 options `|;bsd-auth;index;' [connections stuff deleted] sendbackup-gnutar: time 0.035: doing level 1 dump as listed-incremental from /opt/amanda/amanda5/var/amanda/gnutar-lists/bullcalfkoenigstiger1_3_0 to /opt/amanda/amanda5/var/amanda/gnutar-lists/bullcalfkoenigstiger1_3_1.new sendbackup-gnutar: time 32.756: doing level 1 dump from date: 2006-07-19 16:28:22 GMT sendbackup: time 32.879: spawning /opt/amanda/amanda5/libexec/runtar in pipeline sendbackup: argument list: gtar --create --file - --directory /data/koenigstiger/koenigstiger1/katarina --one-file-system --listed-incremental /opt/amanda/amanda5/var/amanda/gnutar-lists/bullcalfkoenigstiger1_3_1.new --sparse --ignore-failed-read --totals . sendbackup: time 32.883: started index creator: /usr/freeware/bin/tar -tf - 2/dev/null | sed -e 's/^\.//' sendbackup-gnutar: time 32.888: /opt/amanda/amanda5/libexec/runtar: pid 2015710 sendbackup: time 10123.404: index created successfully sendbackup: time 10123.416: 53:size(|): Total bytes written: 49963202560 (46GB, 4.7MB/s) sendbackup: time 10162.761: pid 2017899 finish time Tue Jul 25 22:14:29 2006 thanks jf
Re: more fun with levels and estimates fubar
On Thu, Jul 27, 2006 at 10:26:28AM -0400, Jean-Francois Malouin wrote: Continuing with my estimate saga... What would cause a level 1 backup of a DLE with gnutar be as big as a full? Estimates are done with calcsize and gnutar is 1.13.25 with amanda-2.4.5 Could it be that the gnutar-lists file is corrupted? Looking at them I see that they all have the same size for level 0 1 and 2...Normal? Are the gnutar-lists files current? I.e. are they being updated each night? My similar problem had the system creating *.NEW files, but never renaming them after the dump. So I was always using old list files. But record no (my problem) is not yours. I recall from your original post that you were having problems with same size dumps from DLEs the used gnutar and others that used xfsdump. The latter would not use gnutar-lists so I doubt the overall problem was there. But just in case, what about permissions/ownership on the gnutar-list directories and their parent paths. What about the curinfo files. Are they being updated nightly? The solution to my problem was triggered by a several year-old thread were all levels were the same size. The solution to that posters troubles was to stop running a script that nightly touched or chmod'ed all files on the DLEs. Might you be having a similar situation such that all ls -l or ls -cl timestamps are current thus making it look like they require backing up? -- Jon H. LaBadie [EMAIL PROTECTED] JG Computing 4455 Province Line Road(609) 252-0159 Princeton, NJ 08540-4322 (609) 683-7220 (fax)
New question on levels
Hello list: Just want to know if anyone has ever experienced a case where a particular DLE would get bumped two levels instead of one. For example, let's say that after a level 0, instead of level 1, that DLE would get bumped to level 2. Sorry, I had to ask... / Ronald Vincent Vazquez Senior Unix Systems Administrator Senior Network Manager Christ Tabernacle Church Ministries http://www.ctcministries.org (301) 540-9394 Home (240) 401-9192 Cell For web hosting solutions, please visit: http://www.spherenix.com/
Re: New question on levels
On Wed, Jul 26, 2006 at 10:31:51AM -0400, Ronald Vincent Vazquez wrote: Hello list: Just want to know if anyone has ever experienced a case where a particular DLE would get bumped two levels instead of one. For example, let's say that after a level 0, instead of level 1, that DLE would get bumped to level 2. Sorry, I had to ask... If talking about amanda's planning, it shouldn't happen. I'm pretty sure that amanda does at most, estimates of three dump levels; level 0, the last dump level, and the last level +1. When done manually, the label tags are pretty arbitrary though. You can do a full dump, then call the next dump an incremental level 9. Its contents will be the same as if you called it a level 1. I've seen suggested fixed dump level schedules that bounce around. Some of these are on the dump man pages. For example, from ufsdump the first line of a suggested schedule: SunMonTueWedThuFri Week 1: Full5 5 5 5 3 -- Jon H. LaBadie [EMAIL PROTECTED] JG Computing 4455 Province Line Road(609) 252-0159 Princeton, NJ 08540-4322 (609) 683-7220 (fax)
Re: New question on levels
On Wednesday 26 July 2006 10:31, Ronald Vincent Vazquez wrote: Hello list: Just want to know if anyone has ever experienced a case where a particular DLE would get bumped two levels instead of one. For example, let's say that after a level 0, instead of level 1, that DLE would get bumped to level 2. Sorry, I had to ask... From the latest 20060725/example/amanda.conf: bumpsize 20 Mb # minimum savings (threshold) to bump level 1 - 2 bumppercent 20 # minimum savings (threshold) to bump level 1 - 2 bumpdays 1 # minimum days at each level bumpmult 4 # threshold = bumpsize * bumpmult^(level-1) I could see where a certain set of circumstances might trigger such behavior, but in the case of level 0 to level 2, it seems to me it would have to have already done a valid level 1 to use as its reference. A skip of level 1 is the one I can't quite get my aged wet ram to accept. So me goes off in search of additional caffiene, obviously I haven't had enough so far today. :) -- Cheers, Gene People having trouble with vz bouncing email to me should add the word 'online' between the 'verizon', and the dot which bypasses vz's stupid bounce rules. I do use spamassassin too. :-) Yahoo.com and AOL/TW attorneys please note, additions to the above message by Gene Heskett are: Copyright 2006 by Maurice Eugene Heskett, all rights reserved.
Re: Amanda Errors Levels
On Wed, May 10, 2006 at 09:39:42AM -0700, HUGHES Marilyn F wrote: A previous employee left and I have taken over the daily maintenance of Amanda. Can someone help with some newbie questions? * Is there a place that lists the descriptions of the different levels? For instance, we believe that Level 0 is a full backup, but what are levels 1 through 4? level 1 contains changes since last level 0 level 2 contains changes since last level 1 level 3 contains changes since last level 2 etc. Note, changes includes noting deleted files and permission/ownership changes. * Is there a place that lists what the different error messages actually mean? Such as, why are our Windows backups always STRANGE? What does too many dumper retry mean? For windows, probably you are also getting messages saying some files were not accessible. Windows denies backup to amanda of system files and files currently opened by other applications. You should see those details further down in the report. too many dumper retry sounds like a system was unreachable or kept failing in the middle of the connection. NOTE: We have Amanda running on a Solaris 10 zone (it had to be moved to this zone after our Solaris 8 server crashed). It is backing up other Solaris servers and Windows shares. Thanks for your help (and patience!) Other problem: * We have a backup that has been failing the last few nights but previously worked fine. We have been getting the following messages. I Do you mean an entire amdump run, or more likely one host or one DLE (DiskList Entry). thought the messages meant that some other activity was taking place at the same time (such as a database backup) but that doesn't seem to be the case as the files are created around 23:00 and Amanda tries to back them up about 1:00 am. Can anyone tell why it is failing? 5/10/2006 8:28:53 AM hughesm Date: Wed, 10 May 2006 05:46:53 -0700 (PDT) From: Amanda Backup Account TSamclnt [EMAIL PROTECTED] To: [EMAIL PROTECTED], [EMAIL PROTECTED] Subject: third AMANDA MAIL REPORT FOR May 10, 2006 These dumps were to tape DLT748. The next 2 tapes Amanda expects to used are: DLT749, DLT750. FAILURE AND STRANGE DUMP SUMMARY: majic-dw-p /oracle04/orabkup/atgisprd lev 0 FAILED [/usr/sbin/ufsdump returned 3] FAILED AND STRANGE DUMP DETAILS: /-- majic-dw-p /oracle04/orabkup/atgisprd lev 0 FAILED [/usr/sbin/ufsdump returned 3] sendbackup: start [majic-dw-prod:/oracle04/orabkup/atgisprd level 0] sendbackup: info BACKUP=/usr/sbin/ufsdump sendbackup: info RECOVER_CMD=/bin/gzip -dc |/usr/sbin/ufsrestore -f... - sendbackup: info COMPRESS_SUFFIX=.gz sendbackup: info end | DUMP: Writing 32 Kilobyte records | DUMP: Date of this level 0 dump: Wed May 10 01:37:13 2006 | DUMP: Date of last level 0 dump: the epoch | DUMP: Dumping /dev/rdsk/c2t2B60220044E4d0s3 (majic-dw-prod:/oracle04) to standard output. | DUMP: Mapping (Pass I) [regular files] | DUMP: Mapping (Pass II) [directories] | DUMP: Estimated 1046 blocks (5110.68MB) on 0.08 tapes. | DUMP: Dumping (Pass III) [directories] | DUMP: Dumping (Pass IV) [regular files] | DUMP: 54.62% done, finished in 0:08 | DUMP: Warning - block 1179729400 is beyond the end of `/dev/rdsk/c2t2B60220044E4d0s3' (Numerous errors like these.) ? DUMP: bread: dev_seek error: Error 0 | DUMP: Warning - block 439137164 is beyond the end of `/dev/rdsk/c2t2B60220044E4d0s3' | DUMP: Warning - block 168034346 is beyond the end of `/dev/rdsk/c2t2B60220044E4d0s3' Uncertain -- things I'd check is whether your disk partitions might slightly overlap and whether there are any bugreports as Sun support regarding ufsdump that sound like this. -- Jon H. LaBadie [EMAIL PROTECTED] JG Computing 4455 Province Line Road(609) 252-0159 Princeton, NJ 08540-4322 (609) 683-7220 (fax)
Re: amanda dumping salad of levels
Hi Jon, - I add "strategy nofull" to all incremental configs, and its work fine. - In full backup scripts is amadmin with force option. - All my daily incremental backups is flushing to the same tape. Thank you very much. Leonid Jon LaBadie wrote: On Sun, Jun 26, 2005 at 04:13:30PM +0300, Leonid Shulov wrote: Hi, My amanda configurations is: - DailySetPS - Fraiday weekly to tape (Full dump PS and servers) 00 0 * * 5 root /etc/amanda/amdumpPS.cron - DailySetL - Workday nightly (5 days) to holding disk (incremental dump PS and servers with Mandrake Linux PC) 30 1 * * 0-4 root /etc/amanda/amdump00.cron - DailySetTP - Monday weekly to tape (Full dump notebooks) /etc/amanda/amdumpTP.cron - DailySetY - Thuesday weekly to tape (Full dump boss notebooks) 00 10 * * 0-4 root /etc/amanda/amdump10.cron - DailySetE - Wednesday nightly to tape (Full dump Mandrake Linux PC) 00 20 * * 3 root /etc/amanda/amdumpPE.cron - DailySet2 - Workday (3 times X 5 days) to holding disk (12:00,15:00,18:00 incremental dump notebooks) 00 12 * * 0-4 root /etc/amanda/amdump12.cron 00 15 * * 0-4 root /etc/amanda/amdump15.cron 00 18 * * 0-3 root /etc/amanda/amdump18.cron - DailySetY - Workday (1 time X 5 days) to holding disk (10:00 incremental dump boss notebook) 00 10 * * 0-4 root /etc/amanda/amdump10.cron - DailySetF - Workday nightly (5 days) flush from holding disk to tape 00 20 * * 0-4 root /etc/amanda/amflush.cron Attached: 1. /etc/amanda/DailySet2/amanda.conf 2. /home/amanda/amd/ logfile Jon LaBadie wrote: On Thu, Jun 23, 2005 at 02:10:55PM +0300, Leonid Shulov wrote: Hi, I upgraded my amanda to 2.4.5 version, and after that sometimes amanda do backup with lev0 without any tape to holding disk. This broke all my work order. Before I do full PC dump once a week at Saturday nightly, incremental PC dump nightly in working days to holding disk. Daily incremental Notebook dump I do to holding disk. And now I don't know what to do, amanda dumping salad of levels: 0,1n The behavior you describe, a mixture of levels, is the 'normal' behavior for amanda. That amanda was behaving differently before suggests you had some aspect of your configuration set to force amanda to your desireed scheme. I'm surprised that a version change in the amanda executables would change its behavior with respect to an existing configuration. Let us know how your configuration tries to force amanda into the non-standard behavior you want. Maybe we will be able to notice something that is now apparent from what your posted. Leonid, amazingly complex setup. Seems like there could be a simpler, ?better? way to solve your requirements. But you know your needs better than we. It seems like you run many configs, with perhaps a total of about 40 amump runs per/work week. From what you present I can't tell if there is any overlap in config directories, disklists, log and index files, ... The one you show in more detail, DailySet2, seems to use the a subdir of the amanda user's home for its logs. Also this config uses tapes called DailySet1 despite the DailySet2 name. Do all your configs get flushed to the same tape? None of that addresses your stated problem, a changed behavior where you are now getting a mixture of full and incremental dumps when you want one or the other. But I see nothing in your DailySet2 amanda.conf file that would force one or the other behavior, no "increment only" type directives, so the behavior you are seeing is exactly the behavior I would expect, amanda's default of a mixture of levels. Is there something in your cron scripts that you expect would force this behavior? A common way to force amanda to behave the way you seem to want is to have a very long dumpcycle, a directive of increment only or no-full, and when you actually do want a full dump, use amadmin to "force" a level 0 dump on the next run.
Re: amanda dumping salad of levels
Hi, My amanda configurations is: - DailySetPS - Fraiday weekly to tape (Full dump PS and servers) 00 0 * * 5 root /etc/amanda/amdumpPS.cron - DailySetL - Workday nightly (5 days) to holding disk (incremental dump PS and servers with Mandrake Linux PC) 30 1 * * 0-4 root /etc/amanda/amdump00.cron - DailySetTP - Monday weekly to tape (Full dump notebooks) /etc/amanda/amdumpTP.cron - DailySetY - Thuesday weekly to tape (Full dump boss notebooks) 00 10 * * 0-4 root /etc/amanda/amdump10.cron - DailySetE - Wednesday nightly to tape (Full dump Mandrake Linux PC) 00 20 * * 3 root /etc/amanda/amdumpPE.cron - DailySet2 - Workday (3 times X 5 days) to holding disk (12:00,15:00,18:00 incremental dump notebooks) 00 12 * * 0-4 root /etc/amanda/amdump12.cron 00 15 * * 0-4 root /etc/amanda/amdump15.cron 00 18 * * 0-3 root /etc/amanda/amdump18.cron - DailySetY - Workday (1 time X 5 days) to holding disk (10:00 incremental dump boss notebook) 00 10 * * 0-4 root /etc/amanda/amdump10.cron - DailySetF - Workday nightly (5 days) flush from holding disk to tape 00 20 * * 0-4 root /etc/amanda/amflush.cron Attached: 1. /etc/amanda/DailySet2/amanda.conf 2. /home/amanda/amd/ logfile Jon LaBadie wrote: On Thu, Jun 23, 2005 at 02:10:55PM +0300, Leonid Shulov wrote: Hi, I upgraded my amanda to 2.4.5 version, and after that sometimes amanda do backup with lev0 without any tape to holding disk. This broke all my work order. Before I do full PC dump once a week at Saturday nightly, incremental PC dump nightly in working days to holding disk. Daily incremental Notebook dump I do to holding disk. And now I don't know what to do, amanda dumping salad of levels: 0,1n The behavior you describe, a mixture of levels, is the 'normal' behavior for amanda. That amanda was behaving differently before suggests you had some aspect of your configuration set to force amanda to your desireed scheme. I'm surprised that a version change in the amanda executables would change its behavior with respect to an existing configuration. Let us know how your configuration tries to force amanda into the non-standard behavior you want. Maybe we will be able to notice something that is now apparent from what your posted. # # amanda.conf - sample Amanda configuration file. This started off life as # the actual config file in use at CS.UMD.EDU. # # If your configuration is called, say, csd, then this file normally goes # in /etc/amanda/csd/amanda.conf. # org DailySet2 # your organization name for reports mailto root # space separated list of operators at your siteDailySet1 dumpuser backup # the user to run dumps under inparallel 4# maximum dumpers that will run in parallel netusage 1000 Kbps # maximum net bandwidth for Amanda, in KB per sec dumpcycle 7 days# the number of days in the normal dump cycle runspercycle 15 # the number of amdump runs in dumpcycle days tapecycle 17 tapes # the number of tapes in rotation # 4 weeks (dumpcycle) times 5 tapes per week (just # the weekdays) plus a few to handle errors that # need amflush and so we do not overwrite the full # backups performed at the beginning of the previous # cycle ### ### ### # WANING: don't use `inf' for tapecycle, it's broken! ### ### ### bumpsize 20 Mb # minimum savings (threshold) to bump level 1 - 2 bumpdays 1 # minimum days at each level bumpmult 4 # threshold = bumpsize * bumpmult^(level-1) etimeout 300# number of seconds per filesystem for estimates. #etimeout -600 # total number of seconds for estimates. # a positive number will be multiplied by the number of filesystems on # each host; a negative number will be taken as an absolute total time-out. # The default is 5 minutes per filesystem. # Specify tape device and/or tape changer. If you don't have a tape # changer, and you don't want to use more than one tape per run of # amdump, just comment out the definition of tpchanger. # Some tape changers require tapedev to be defined; others will use # their own tape device selection mechanism. Some use a separate tape # changer device (changerdev), others will simply ignore this # parameter. Some rely on a configuration file (changerfile) to # obtain more information about tape devices, number of slots, etc; # others just need to store some data in files, whose names will start # with changerfile. For more information about individual tape # changers, read docs/TAPE.CHANGERS. # At most one changerfile entry must be defined; select the most # appropriate one for your configuration. If you select man-changer, # keep the first one; if you decide not to use a tape changer, you may # comment them all out. runtapes 1 # number of tapes to be used in a single run
Re: amanda dumping salad of levels
On Sun, Jun 26, 2005 at 04:13:30PM +0300, Leonid Shulov wrote: Hi, My amanda configurations is: - DailySetPS - Fraiday weekly to tape (Full dump PS and servers) 00 0 * * 5 root /etc/amanda/amdumpPS.cron - DailySetL - Workday nightly (5 days) to holding disk (incremental dump PS and servers with Mandrake Linux PC) 30 1 * * 0-4 root /etc/amanda/amdump00.cron - DailySetTP - Monday weekly to tape (Full dump notebooks) /etc/amanda/amdumpTP.cron - DailySetY - Thuesday weekly to tape (Full dump boss notebooks) 00 10 * * 0-4 root /etc/amanda/amdump10.cron - DailySetE - Wednesday nightly to tape (Full dump Mandrake Linux PC) 00 20 * * 3 root /etc/amanda/amdumpPE.cron - DailySet2 - Workday (3 times X 5 days) to holding disk (12:00,15:00,18:00 incremental dump notebooks) 00 12 * * 0-4 root /etc/amanda/amdump12.cron 00 15 * * 0-4 root /etc/amanda/amdump15.cron 00 18 * * 0-3 root /etc/amanda/amdump18.cron - DailySetY - Workday (1 time X 5 days) to holding disk (10:00 incremental dump boss notebook) 00 10 * * 0-4 root /etc/amanda/amdump10.cron - DailySetF - Workday nightly (5 days) flush from holding disk to tape 00 20 * * 0-4 root /etc/amanda/amflush.cron Attached: 1. /etc/amanda/DailySet2/amanda.conf 2. /home/amanda/amd/ logfile Jon LaBadie wrote: On Thu, Jun 23, 2005 at 02:10:55PM +0300, Leonid Shulov wrote: Hi, I upgraded my amanda to 2.4.5 version, and after that sometimes amanda do backup with lev0 without any tape to holding disk. This broke all my work order. Before I do full PC dump once a week at Saturday nightly, incremental PC dump nightly in working days to holding disk. Daily incremental Notebook dump I do to holding disk. And now I don't know what to do, amanda dumping salad of levels: 0,1n The behavior you describe, a mixture of levels, is the 'normal' behavior for amanda. That amanda was behaving differently before suggests you had some aspect of your configuration set to force amanda to your desireed scheme. I'm surprised that a version change in the amanda executables would change its behavior with respect to an existing configuration. Let us know how your configuration tries to force amanda into the non-standard behavior you want. Maybe we will be able to notice something that is now apparent from what your posted. Leonid, amazingly complex setup. Seems like there could be a simpler, ?better? way to solve your requirements. But you know your needs better than we. It seems like you run many configs, with perhaps a total of about 40 amump runs per/work week. From what you present I can't tell if there is any overlap in config directories, disklists, log and index files, ... The one you show in more detail, DailySet2, seems to use the a subdir of the amanda user's home for its logs. Also this config uses tapes called DailySet1 despite the DailySet2 name. Do all your configs get flushed to the same tape? None of that addresses your stated problem, a changed behavior where you are now getting a mixture of full and incremental dumps when you want one or the other. But I see nothing in your DailySet2 amanda.conf file that would force one or the other behavior, no increment only type directives, so the behavior you are seeing is exactly the behavior I would expect, amanda's default of a mixture of levels. Is there something in your cron scripts that you expect would force this behavior? A common way to force amanda to behave the way you seem to want is to have a very long dumpcycle, a directive of increment only or no-full, and when you actually do want a full dump, use amadmin to force a level 0 dump on the next run. -- Jon H. LaBadie [EMAIL PROTECTED] JG Computing 4455 Province Line Road(609) 252-0159 Princeton, NJ 08540-4322 (609) 683-7220 (fax)
amanda dumping salad of levels
Hi, I upgraded my amanda to 2.4.5 version, and after that sometimes amanda do backup with lev0 without any tape to holding disk. This broke all my work order. Before I do full PC dump once a week at Saturday nightly, incremental PC dump nightly in working days to holding disk. Daily incremental Notebook dump I do to holding disk. And now I don't know what to do, amanda dumping salad of levels: 0,1n Thanks, Leonid
Re: amanda dumping salad of levels
On Thu, Jun 23, 2005 at 02:10:55PM +0300, Leonid Shulov wrote: Hi, I upgraded my amanda to 2.4.5 version, and after that sometimes amanda do backup with lev0 without any tape to holding disk. This broke all my work order. Before I do full PC dump once a week at Saturday nightly, incremental PC dump nightly in working days to holding disk. Daily incremental Notebook dump I do to holding disk. And now I don't know what to do, amanda dumping salad of levels: 0,1n The behavior you describe, a mixture of levels, is the 'normal' behavior for amanda. That amanda was behaving differently before suggests you had some aspect of your configuration set to force amanda to your desireed scheme. I'm surprised that a version change in the amanda executables would change its behavior with respect to an existing configuration. Let us know how your configuration tries to force amanda into the non-standard behavior you want. Maybe we will be able to notice something that is now apparent from what your posted. -- Jon H. LaBadie [EMAIL PROTECTED] JG Computing 4455 Province Line Road(609) 252-0159 Princeton, NJ 08540-4322 (609) 683-7220 (fax)
Levels
I'm still having a hard time figuring out how levels work. Can someone confirm to me that each time amanda is run, it will backup a file if it has been modified during the day (or since the last run) In other words: does amanda guarantee that if a file is modified after a dump, it will be included in the next dump, whatever the level is used ? Thanks
Re: Levels
On Thu, 12 May 2005, Guy Dallaire wrote: I'm still having a hard time figuring out how levels work. Can someone confirm to me that each time amanda is run, it will backup a file if it has been modified during the day (or since the last run) In other words: does amanda guarantee that if a file is modified after a dump, it will be included in the next dump, whatever the level is used ? No. Amanda guarantees it will take all data your backup program sends it, put it onto the tape, and get it back off again, assuming the tape is still good at that time. Whether or not a modified file gets included in the next dump is the responsibility of whatever backup program you tell amanda to invoke. Typically dump or gnutar, but folks have managed to glue in a number of others as well. Amanda doesn't do backups, she only oversees the process. -Mitch
Re: Levels
On Thu, May 12, 2005 at 03:28:37PM -0400, Guy Dallaire enlightened us: I'm still having a hard time figuring out how levels work. Can someone confirm to me that each time amanda is run, it will backup a file if it has been modified during the day (or since the last run) In other words: does amanda guarantee that if a file is modified after a dump, it will be included in the next dump, whatever the level is used ? Yes, any modified file will be included. I posted a few days ago my attempt at explaining levels. You might re-read that and see if it makes any more sense. If you want me to try to explain it in a different way, I can, or stop by the #amanda IRC channel and we can talk in real time. Matt (a.k.a. euclid) -- Matt Hyclak Department of Mathematics Department of Social Work Ohio University (740) 593-1263 pgp4FMMzmQWNi.pgp Description: PGP signature
Re: Levels
On Thu, 12 May 2005, Guy Dallaire wrote:
I'm still having a hard time figuring out how levels work.
Can someone confirm to me that each time amanda is run, it will backup
a file if it has been modified during the day (or since the last run)
Yes.
In other words: does amanda guarantee that if a file is modified after
a dump, it will be included in the next dump, whatever the level is
used ?
Yes. Amanda always backs up all modified files.
The question is: does she always back up all _unmodified_ files? Here the
answer is: no, only when doing a level 0 backup.
Level 0 means a full backup (all modified and unmodified files).
Level n (n 0) means a backup of all files that were modified since the last
level n-1.
Hope this helps...
Gr{oetje,eeting}s,
Geert
--
Geert Uytterhoeven -- There's lots of Linux beyond ia32 -- [EMAIL PROTECTED]
In personal conversations with technical people, I call myself a hacker. But
when I'm talking to journalists I just say programmer or something like that.
-- Linus Torvalds
Re: Levels
[Please reply to amanda-users when asking questions on the list] On Thu, 12 May 2005, Guy Dallaire wrote: 2005/5/12, Mitch Collinsworth [EMAIL PROTECTED]: No. Amanda guarantees it will take all data your backup program sends it, put it onto the tape, and get it back off again, assuming the tape is still good at that time. Whether or not a modified file gets included in the next dump is the responsibility of whatever backup program you tell amanda to invoke. Typically dump or gnutar, but folks have managed to glue in a number of others as well. Amanda doesn't do backups, she only oversees the process. -Mitch Well... What I meant is that, provided I have a standard dump cycle, I use gnutar for some DLE's, and dump for other DLE's, and I run amanda at time x. Will amanda run dump or gnu-tar in a way that if a file is modified before time x, it will ALWAYS be included in the backup ? ALWAYS is a very strong word. I see others have been responding yes to this but nevertheless the answer is still no. Amanda makes a very good effort to always do the right thing but there are various ways in which things can still go wrong. The best you can really say is that amanda will ALWAYS try. For example if you have way too much data to fit on the tape, sometimes amanda will say [dumps way too big, must skip incremental dumps]. Now this is generally a very rare occurance and typically one that is not caused by any failing in amanda, only that you gave her more to do than is physically possible and she had to decide SOMETHING. Also note that there are ways to configure your amanda setup to make this highly unlikely. For example a tape loader with plenty of tapes available and a configuration that tells amanda she can use multiple tapes per run if necessary. etc. -Mitch
Re: Levels
On Thu, May 12, 2005 at 05:58:02PM -0400, Mitch Collinsworth wrote: On Thu, 12 May 2005, Guy Dallaire wrote: What I meant is that, provided I have a standard dump cycle, I use gnutar for some DLE's, and dump for other DLE's, and I run amanda at time x. Will amanda run dump or gnu-tar in a way that if a file is modified before time x, it will ALWAYS be included in the backup ? ALWAYS is a very strong word. I see others have been responding yes to this but nevertheless the answer is still no. Amanda makes a very good effort to always do the right thing but there are various ways in which things can still go wrong. The best you can really say is that amanda will ALWAYS try. For example if you have way too much data to fit on the tape, sometimes amanda will say [dumps way too big, must skip incremental dumps]. Now this is generally a very rare occurance and typically one that is not caused by any failing in amanda, only that you gave her more to do than is physically possible and she had to decide SOMETHING. I had a similar reaction to Mitch. And thus I'll just pass on some other examples of how always should never be specified :) Suppose the file(s) in question were on a file system or a laptop that at the time of backup were not available. A client of mine changed the system clock by a week, then by another week to avoid a license expiration. Really messed with the backups. Windows will not let you copy or backup a file already open for use by another application. So if it was modified, but in use, no backup. Note, these are all problems outside of amanda's control. And of the various backup programs amanda relies upon. But to answer your real concern, given the use of standard dump's and guntar, a file modified since the last run of amdump 'should' be in the backup file of the next run of amdump. Ahh, I just thought of an exception; I have two DiskListEntries that are for directories that almost never change. One is only CD images. I have set them up to skip-incrementals and only do level 0, full dumps of them every two weeks. If anything changes on the 13 days between full dumps I would not have a backup copy of it. But that is because I chose to not have a copy. -- Jon H. LaBadie [EMAIL PROTECTED] JG Computing 4455 Province Line Road(609) 252-0159 Princeton, NJ 08540-4322 (609) 683-7220 (fax)
backup levels clarification needed
Hi all, I have a working amanda setup. I use a 14d dumpcycle and 20 tapecycle with daily backups. I just want to conmfirm my understanding: When issuing an amovervie I get the following (excuse the text wrap) date 12 12 12 12 12 12 12 12 12 01 01 01 01 01 01 01 01 01 01 01 host disk 23 24 25 26 27 28 29 30 31 01 02 03 04 05 06 07 08 09 10 11 alphacen /home 2 3 4 4 5 5 5 5 5 5 5 5 0 1 2 3 3 3 3 4 does this mean that if I want a complete restore of /home, I can only get one from not before 01/04 ? -- As I see it I do not have a previous level 0 anymore, so before 01/04 I have incomplete data Shai
Re: backup levels clarification needed
Shai Ayal wrote: Hi all, I have a working amanda setup. I use a 14d dumpcycle and 20 tapecycle with daily backups. I just want to conmfirm my understanding: When issuing an amovervie I get the following (excuse the text wrap) date 12 12 12 12 12 12 12 12 12 01 01 01 01 01 01 01 01 01 01 01 host disk 23 24 25 26 27 28 29 30 31 01 02 03 04 05 06 07 08 09 10 11 alphacen /home 2 3 4 4 5 5 5 5 5 5 5 5 0 1 2 3 3 3 3 4 does this mean that if I want a complete restore of /home, I can only get one from not before 01/04 ? -- As I see it I do not have a previous level 0 anymore, so before 01/04 I have incomplete data Yes. Completely correct. If you need more, and you have the tape capacity, lower the dumpcycle to e.g. 7 days. Having more than one level 0 is better anyway. -- Paul Bijnens, XplanationTel +32 16 397.511 Technologielaan 21 bus 2, B-3001 Leuven, BELGIUMFax +32 16 397.512 http://www.xplanation.com/ email: [EMAIL PROTECTED] *** * I think I've got the hang of it now: exit, ^D, ^C, ^\, ^Z, ^Q, F6, * * quit, ZZ, :q, :q!, M-Z, ^X^C, logoff, logout, close, bye, /bye, * * stop, end, F3, ~., ^]c, +++ ATH, disconnect, halt, abort, hangup, * * PF4, F20, ^X^X, :D::D, KJOB, F14-f-e, F8-e, kill -1 $$, shutdown, * * kill -9 1, Alt-F4, Ctrl-Alt-Del, AltGr-NumLock, Stop-A, ...* * ... Are you sure? ... YES ... Phew ... I'm out * ***
Amanda doesn't count non-activity correctly when bumping levels
Hi Everyone, I have some of my largest volumes that do not have activity for several weeks, and they are also taking the most of my tapes -- even the incrementals. I just did a zdiff on the different indexes for one of these DLEs and they are exactly the same. It seems that the level is not being bumped because there has been no activity. I had bumpsize at 10MB with the bumpdays 1 and bumpmult 1, but the problem still persists. I'm going to try bumpsize at 1 byte to see if it will bump the level because of directory atime updates. I have many volumes where their size is just a few GB smaller than my 35GB (uncompressed) AIT1 tapes, and I'm trying to get Amanda to schedule level 0 dumps on different days (10 tapes and dumpcycle of 7 days), but it can't do that if it won't bump the levels on the inactive volumes. What else can I do? Thanks, Mike
Re: Amanda backup levels
Jeroen Heijungs wrote: Probably very stupid questions, but can someone shed some light on very simple questions: what exactly are the differences between the different levels of backup? I searched the documentation and arcives but so far I have not found the exact definitions. Level n backup is a backup where you save what has changed since the last level n backup. period. As I understand: - level 0 is full-backup - level 1 is incremental backup, based on modification time (?) - what are the higher levels ??? You can do L0 backups once a month, L1 backups once a week, and L2 backups everyday. So, everyday, you will save what you changed since the last L2 backup, that is to say since the day before. I explain that though now I hate incremental backups and pray for not having to do with them anymore. They may look fun (and indeed they DO are very well managed by Amanda) but it may turn to nightmare when you have to restore some directories with changes in different level backups, and one of you tape fails... I ask now, because untill now we only had to backup FreeBSD servers acting as routers, gateways. firewalls, mailrelayservers etc. None of them had real fast-changing user-data, so up-to-date restores were not an high-priority issue and we never had to restore something (rock-solid os!). But now we are moving to imap-mail and that means that there are lots of user-data which change very often and have to restore more often (users delete mail-folders accidently), so I have to be sure that all changed data is backed-up every run, users do not want mails missing. In my company, we use OpenBSD as an imap server, and FreeBSD as a samba server, and the files are changing very fast. So we do L0 backups everynight. If you can afford, avoid incremental backups. Last question: - are there gui-frontends for the restore, that work for the helpdesk staff (they cannont work with the amrestore interface) ? No. I heard they were people working on a web frontend, but hey, when you restore some data, are you sure you will have a gui running ? Are you even sure you will have any network working ? The command line interface is simple enough to be usable in the worst situation, think about that. PS. CALLING ALL CARS : Are the guys working on the web frontend reading this e-mail ? -- Nicolas Ecarnot
Re: Amanda backup levels
Nicolas Ecarnot wrote: Today, I speak to myself : PS. CALLING ALL CARS : Are the guys working on the web frontend reading this e-mail ? http://subwiki.honeypot.net/cgi-bin/view/Computing/AmControl Not tested... -- Nicolas Ecarnot
Re: Amanda backup levels
I asked about the differences between the levels of backup and I got (sofar) two different answers (thanks to you both), so I still do not know for sure: Nicolas Ecarnot wrote: Level n backup is a backup where you save what has changed since the last level n backup. period. Gertjan van Oosten wrote: A level N backup includes all files modified since the last level M backup where M N. So a level 1 backup includes all files changed since the last level 0 backup; a level 2 backup includes all files changed since the last level 1 backup (or if there was no such backup, it includes all files changed since the last level 0 backup), etc. A level 0 backup includes all files, period. That are two different ways, the first I normally call INCREMENTAL, the second I normally call DIFFERENTIAL. I prefer the second, because mostly you will not need every tape, but only the first (full) and the last, and not all others in between. Some more questions: - in theory you can get an infinite level (no limit) backup ? - the more low-level backups you see (0 and 1), the better Amanda can handle the dumpcycle and tape capacity ? Or the other way round, Amanda will try to make as much level 0 backups as possible but based on the dumpcycle and capacity she will choose to do more level 1,2,3,4 backups in between ? tia Jeroen Heijungs Het Muziektheater Amsterdam
Re: Amanda backup levels
Hi Jeroen, As quoted from Jeroen Heijungs [EMAIL PROTECTED]: what exactly are the differences between the different levels of backup? A level N backup includes all files modified since the last level M backup where M N. So a level 1 backup includes all files changed since the last level 0 backup; a level 2 backup includes all files changed since the last level 1 backup (or if there was no such backup, it includes all files changed since the last level 0 backup), etc. A level 0 backup includes all files, period. See e.g. the manual page for ufsdump for some more information. And to be sure: - at least once per dumpcycle there is a fullbackup Amanda tries to do that, yes. You will get a warning if you are near the end of your tape cycle and a level 0 dump of a file system hasn't been made thus far. [Last full dump of %s:%s on tape %s overwritten in %d run%s.] - the other runs are mostly level 1 (incremental), That depends. Amanda tries to balance things and if there's enough room on the tape, it may decide to run a level 0 more than once in the tape cycle. so all changes to the filesystems are on tape - is this right ??? In principle (unless your tape cycle is too short, or you have too much incremental data to fit even on one tape) all changed files are somewhere in the tape cycle. Met vriendelijke groet, -- -- Gertjan van Oosten, [EMAIL PROTECTED], West Consulting B.V., +31 15 2191 600
Re: Amanda backup levels
Hi Jeroen, As quoted from Jeroen Heijungs [EMAIL PROTECTED]: I asked about the differences between the levels of backup and I got (sofar) two different answers (thanks to you both), so I still do not know for sure: My answer was paraphrased from the ufsdump documentation. Some more questions: - in theory you can get an infinite level (no limit) backup ? In theory: yes. In practice: ufsdump limits the levels to 0 through 9. Kind regards, -- -- Gertjan van Oosten, [EMAIL PROTECTED], West Consulting B.V., +31 15 2191 600
Re: Amanda backup levels
Jeroen Heijungs wrote: I asked about the differences between the levels of backup and I got (sofar) two different answers (thanks to you both), so I still do not know for sure: Nicolas Ecarnot wrote: Level n backup is a backup where you save what has changed since the last level n backup. period. Gertjan van Oosten wrote: A level N backup includes all files modified since the last level M backup where M N. So a level 1 backup includes all files changed since the last level 0 backup; a level 2 backup includes all files changed since the last level 1 backup (or if there was no such backup, it includes all files changed since the last level 0 backup), etc. A level 0 backup includes all files, period. Indeed, the M N statement is correct. You can start with a L0 backup once a year, then do one L1 backup each month, then do a L2 backup everyday. Everyday, what will be saved is what changed since the start of the month. So normally, on april 1th, the L1 backup you'll do will contain what changed since the start of the year. From now, I'm less sure of what I say On 20040402, to restore a complete system that was set up on january 1th and crashed today, you'll have to provide : - L0 2004 - L1 200404 - L2 20040402 /not sure Experts, please correct me. (As you can see, I prefer level 0 backups ;o) Some more questions: - in theory you can get an infinite level (no limit) backup ? Yes. tia Salut. -- Nicolas Ecarnot
Re: Amanda backup levels
Hi Nicolas, As quoted from Nicolas Ecarnot [EMAIL PROTECTED]: On 20040402, to restore a complete system that was set up on january 1th and crashed today, you'll have to provide : - L0 2004 - L1 200404 - L2 20040402 Correct. Salut, -- -- Gertjan van Oosten, [EMAIL PROTECTED], West Consulting B.V., +31 15 2191 600
Re: Amanda backup levels
On Wed, Apr 21, 2004 at 10:03:24AM +0200, Nicolas Ecarnot wrote: Level n backup is a backup where you save what has changed since the last level n backup. period. Minor correction: A level N backup saves all files that have changed since the last level N-1 backup. Level 0 gets everything, since there are no level -1s. The practical effect of this is that restoring will require at most one tape more than the highest-level backup that has been run. (e.g., If a DLE gets up to level 3 (the highest I've seen here), restoring it will require, at most, 4 tapes (one each of L0, L1, L2, and L3) regardless of the dumpcycle or how many tapes of each level have been run.)
RE: Dumps Levels 1?
Thank you for the suggestions. I'll checkout the bumpsize, bumpmult, and bumpdays report my findings. -Rob -Original Message- From: Jon LaBadie [mailto:[EMAIL PROTECTED] Sent: Tuesday, September 02, 2003 12:47 PM To: [EMAIL PROTECTED] Subject: Re: Dumps Levels 1? On Tue, Sep 02, 2003 at 11:36:40AM -0500, Chris Johnson wrote: you'll need to work with the bumpsize and bumpmult entries in amanada.conf. mine are set to 100Mb and 2 respectivly. This seems to give me level 3 or 4 on my heavily used filesystems for most of the week. bumpdays may be a factor here as well. -- Jon H. LaBadie [EMAIL PROTECTED] JG Computing 4455 Province Line Road(609) 252-0159 Princeton, NJ 08540-4322 (609) 683-7220 (fax)
Dumps Levels 1?
My config: Redhat 9.0 Amanda version 2.4.4p1 I'm slightly concerned with my backups. I'm backingup a central NFS server for my UNIX network. Using disklist, I broke the NFS share into 21 separate sections. I did this so that I could benefit from higher dump levels on sections that are used less. I've been running amanda for the past 2 weeks, but only have dump levels of zero, or one. I ran 2 backups over labor day weekend, and _STILL_ am unable to achieve a dump level higher than 1. And the amreports show no dump level bump adjustments. I guess my question is why isn't amanda using higher dump levels on sections where the data hasn't changed? -Rob
Re: Dumps Levels 1?
Dege, Robert C. wrote: I've been running amanda for the past 2 weeks, but only have dump levels of zero, or one. I ran 2 backups over labor day weekend, and _STILL_ am unable to achieve a dump level higher than 1. And the amreports show no dump level bump adjustments. I guess my question is why isn't amanda using higher dump levels on sections where the data hasn't changed? If the data doesn't change, then why would it need to increase the dumplevel above 1? Or are you indicating that the dumpsize now is considerably more than what is indicated in amadmin Config bumpsize? -- Paul Bijnens, XplanationTel +32 16 397.511 Technologielaan 21 bus 2, B-3001 Leuven, BELGIUMFax +32 16 397.512 http://www.xplanation.com/ email: [EMAIL PROTECTED] *** * I think I've got the hang of it now: exit, ^D, ^C, ^\, ^Z, ^Q, F6, * * quit, ZZ, :q, :q!, M-Z, ^X^C, logoff, logout, close, bye, /bye, * * stop, end, F3, ~., ^]c, +++ ATH, disconnect, halt, abort, hangup, * * PF4, F20, ^X^X, :D::D, KJOB, F14-f-e, F8-e, kill -1 $$, shutdown, * * kill -9 1, Alt-F4, Ctrl-Alt-Del, AltGr-NumLock, Stop-A, ...* * ... Are you sure? ... YES ... Phew ... I'm out * ***
Re: Dumps Levels 1?
you'll need to work with the bumpsize and bumpmult entries in amanada.conf. mine are set to 100Mb and 2 respectivly. This seems to give me level 3 or 4 on my heavily used filesystems for most of the week. Jon LaBadie wrote: On Tue, Sep 02, 2003 at 08:09:20AM -0700, Dege, Robert C. wrote: My config: Redhat 9.0 Amanda version 2.4.4p1 I'm slightly concerned with my backups. I'm backingup a central NFS server for my UNIX network. Using disklist, I broke the NFS share into 21 separate sections. I did this so that I could benefit from higher dump levels on sections that are used less. I've been running amanda for the past 2 weeks, but only have dump levels of zero, or one. I ran 2 backups over labor day weekend, and _STILL_ am unable to achieve a dump level higher than 1. And the amreports show no dump level bump adjustments. I guess my question is why isn't amanda using higher dump levels on sections where the data hasn't changed? When it calculates the size of the next dump level it doesn't feel it saves a sufficient amount of tape/size/time/??? to warrent bumping the level. I've never played with them, but there are a couple of parameters that affect what is sufficient. My systems seldom do level 2's either. Of course that makes recovery simpler. Two tapes rather than three.
Re: Dumps Levels 1?
On Tue, Sep 02, 2003 at 11:36:40AM -0500, Chris Johnson wrote: you'll need to work with the bumpsize and bumpmult entries in amanada.conf. mine are set to 100Mb and 2 respectivly. This seems to give me level 3 or 4 on my heavily used filesystems for most of the week. Jon LaBadie wrote: On Tue, Sep 02, 2003 at 08:09:20AM -0700, Dege, Robert C. wrote: My config: Redhat 9.0 Amanda version 2.4.4p1 I'm slightly concerned with my backups. I'm backingup a central NFS server for my UNIX network. Using disklist, I broke the NFS share into 21 separate sections. I did this so that I could benefit from higher dump levels on sections that are used less. I've been running amanda for the past 2 weeks, but only have dump levels of zero, or one. I ran 2 backups over labor day weekend, and _STILL_ am unable to achieve a dump level higher than 1. And the amreports show no dump level bump adjustments. I guess my question is why isn't amanda using higher dump levels on sections where the data hasn't changed? When it calculates the size of the next dump level it doesn't feel it saves a sufficient amount of tape/size/time/??? to warrent bumping the level. I've never played with them, but there are a couple of parameters that affect what is sufficient. My systems seldom do level 2's either. Of course that makes recovery simpler. Two tapes rather than three. bumpdays may be a factor here as well. -- Jon H. LaBadie [EMAIL PROTECTED] JG Computing 4455 Province Line Road(609) 252-0159 Princeton, NJ 08540-4322 (609) 683-7220 (fax)
Re: Simple question on levels and bumping
"Dirk Webster" [EMAIL PROTECTED] writes: Hi all Thanks to previous help from this list, my first week with amanda has passed smoothly. I'm still a little bemused however over what levels are all about. I understand level 0 is a full backup but what happens in a level 1 or 2 backup. Is it 'things that have changed' plus a bit? A level 1 dump does backup of everything that changed since last level 0 dump. Level 2 does all since last level 1 and so on. Regards Jens Bech Madsen -- Jens Bech Madsen The Stibo Group, Denmark
Levels
Hi, people In order to perform my 22GB (level 0) i have to use a SONY DDS3, but whenever i need to backup everything i have to use that "strategy nofull" parameter to perferm incremental backups and then i use "admin XXX force " to force level 0 backups of the filesystem i want, doing that i can control wich filesystems will be backed up that day on level 0 and doing it allows me to arrange my backup and make it fit my DDS. The main problem is that when i use "strategy nofull" amanda is only performing level 1 backups. I once read something about a patch that could allow me to control the backup levels in the way i want to, is there such a thing ?? Another question, does anybody know if there is any problem on performing (forcing) a level 0 backup and in the next 3 days let amanda perform only level 1 backups, will this screw my backup integrity, do you think i will be able to restore the entire backup ?? Thanks in advanced, Diogo Batista Salgueiro
Re: Levels
[EMAIL PROTECTED] writes: Hi, people In order to perform my 22GB (level 0) i have to use a SONY DDS3, but whenever i need to backup everything i have to use that "strategy nofull" parameter to perferm incremental backups and then i use "admin XXX force " to force level 0 backups of the filesystem i want, doing that i can control wich filesystems will be backed up that day on level 0 and doing it allows me to arrange my backup and make it fit my DDS. The main problem is that when i use "strategy nofull" amanda is only performing level 1 backups. I once read something about a patch that could allow me to control the backup levels in the way i want to, is there such a thing ?? Another question, does anybody know if there is any problem on performing (forcing) a level 0 backup and in the next 3 days let amanda perform only level 1 backups, will this screw my backup integrity, do you think i will be able to restore the entire backup ?? Thanks in advanced, Diogo Batista Salgueiro Diogo, I don't understand what you are trying to achieve. So I'll give some standard answers: Case 1: Your 22 GB are on one physical partition and don't fit on one tape. The workaround is let tar do the backups and to put several sub directories as entries in the disk list and to use an exclude list for the subdirectory entries: #old: #server / std dumptype #new: server /usrstd dumptype server /home std dumptype server / custom dumptype The exclude list in the custom dumptype excludes /usr and /home. Case 2: Even after Amanda has leveled out the tape usage after several dump cycles (if left alone without forces and wrong tapes) you need 2 tapes per day. You could just set tapes per day to 2 and amflush the second tape the next morning. Or you fiddle with the manual changer script. Or you could get a bigger tape format and/or a tape changer. Case 3: You're trying to be smarter than Amanda. She does not like that :-) Amanda schedules backup levels on its own to keep the usage of all tapes at the same level (e.g. all tapes 70% full). If you need more level 0's on fewer tapes you can reduce the dumpcycle parameter and use the same nuber of tapes as of now. So in all your tapes you might have several level 0's of the same file system. HTH, Johannes Nieß
how can I know which levels?
Hi people, How can I check which levels of Amanda will made in backup tape? (what kind of command i can use to do that? exist any?) Thanks in advanced, Diogo Batista Salgueiro Curitiba - Paran - Brasil
Re: how can I know which levels?
How can I check which levels of Amanda will made in backup tape? (what kind of command i can use to do that? exist any?) It's difficult to know what level Amanda will use in the next run. That depends on how much data there is to be backed up. If you want to know what levels it did on an already written tape, you can use amtoc or the find subcommand of amadmin. Diogo Batista Salgueiro John R. Jackson, Technical Software Specialist, [EMAIL PROTECTED]
RE: To force backups in different levels
Diogo, One of the advantages of AMANDA is that she automatically selects the backup level that is needed based on the schedule you specify. If you really need to tweak backup levels, the amadmin command can do this, with the various 'force' options. Check out the amadmin man page for more details. Also, there is a separate list, [EMAIL PROTECTED], for support and configuration questions. You will usually get a faster response to questions there. [EMAIL PROTECTED] is used by developers and bug hunters. I have posted this reply to that list, so we can continue the thread there if necessary. Paul -Original Message- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED]] Sent: Thursday, January 18, 2001 6:40 AM To: [EMAIL PROTECTED] Subject: To force backups in different levels Hello everybody, Does anybody know if there is a way in amanda to force backups in different levels whenever i want ?? ie. let's imagine that, for some reason, i need to make a level 2 backup on a specific machine in my network, how can i do it ?? I heard of some patch (hack) that would allow me to do it, but unfortunately i was unable to find it. Does anyone has it and can send it to me ?? Thank you so much for your time. Diogo Batista Salgueiro Curitiba - Paran - Brasil
new 2.4.2 install; planner/dump levels problems
I've run amanda for some time and just upgraded the server to 2.4.2
(I've left the clients at 2.4.1p1; is there any harm in leaving them
there?).
The server is an Ultra 150 running solaris 2.6. I have a DLT 4700
7 tape stacker, and I use stc-changer to change tapes (which seems to
be working fine). I am not currently using hardware compression. I
have about 100 gb total disk in use to back up.
dumpcycle 2 weeks
runspercycle 12
tapecycle 17 tapes
bumpsize 20 Mb
bumpdays 1
bumpmult 4
Are the relevant settings out of amanda.conf
The dumptypes in use are:
define dumptype comp-user {
global
comment "Non-root partitions on reasonably fast machines"
compress client fast
index
priority medium
}
define dumptype comp-root {
global
comment "Root partitions with compression"
compress client fast
index
priority low
}
Tapetype is DLT:
define tapetype DLT {
comment "DLT tape drives"
length 2 mbytes # 20 Gig tapes
filemark 2000 kbytes# I don't know what this means
speed 1536 kbytes # 1.5 Mb/s
}
The first run, amanda took:
Run Time (hrs:min)15:33
Output Size (meg) 10163.9 9792.0 371.9
Original Size (meg) 30129.728581.9 1547.8
and ran out of tape in some places (which I expected):
***
FAILURE AND STRANGE DUMP SUMMARY:
centipede c0t0d0s6 lev 0 FAILED [out of tape]
bull /dev/md/dsk/d1 lev 1 FAILED [no more holding disk space]
godel /dev/md/dsk/d0 lev 1 FAILED [no more holding disk space]
tajmahal oraclelv02 lev 1 FAILED [no more holding disk space]
pong oraclelv03 lev 1 FAILED [no more holding disk space]
scribble /dev/md/dsk/d80 lev 1 FAILED [no more holding disk space]
moby /dev/md/dsk/d3 lev 1 FAILED [no more holding disk space]
moby /dev/md/dsk/d1 lev 1 FAILED [no more holding disk space]
***
I did a flush of the level 1's it saved on the holding disk.
All is well so far, if not optimal (wish I'd fed her more tape).
The second run, strange things happened:
***
These dumps were to tapes DS1-03, DS1-04.
*** A TAPE ERROR OCCURRED: [[writing file: Bad file number]].
Some dumps may have been left in the holding disk.
Run amflush to flush them to tape.
The next 2 tapes Amanda expects to used are: a new tape, a new tape.
FAILURE AND STRANGE DUMP SUMMARY:
scribble /dev/md/dsk/d80 lev 0 FAILED ["data write: Broken pipe"]
scribble /dev/md/dsk/d80 lev 0 FAILED [dump to tape failed]
escher /dev/md/dsk/d1 lev 0 FAILED [out of tape]
escher /dev/md/dsk/d1 lev 0 FAILED ["data write: Broken pipe"]
escher /dev/md/dsk/d1 lev 0 FAILED [dump to tape failed]
bull /dev/md/dsk/d1 lev 0 FAILED [out of tape]
bull /dev/md/dsk/d1 lev 0 FAILED ["data write: Broken pipe"]
bull /dev/md/dsk/d1 lev 0 FAILED [dump to tape failed]
bull c0t11d0s6 lev 0 FAILED [out of tape]
bull c0t11d0s6 lev 0 FAILED ["data write: Broken pipe"]
bull c0t11d0s6 lev 0 FAILED [dump to tape failed]
planner: Incremental of scribble:c1t2d0s6 bumped to level 2.
planner: Incremental of tajmahal:oraclelv01 bumped to level 2.
planner: Incremental of moby:/dev/md/dsk/d2 bumped to level 2.
planner: Incremental of escher:/dev/md/dsk/d2 bumped to level 2.
planner: Incremental of escher:/dev/md/dsk/d1 bumped to level 2.
taper: tape DS1-03 kb 9980704 fm 80 writing file: short write
taper: retrying scribble:/dev/md/dsk/d80.0 on new tape: [writing
file: short
write]
taper: tape DS1-04 kb 9997120 fm 5 writing file: short write
***
The planner says it's going to do a level 2... but it ACTUALLY tries
to do level 0 on those partitions. Of course, it runs out of tape.
I would expect planner to correctly assess the amount of tape needed
and schedule appropriate levels.
Secondly, it took a phenomenally long time:
Run Time (hrs:min)34:06
Output Size (meg) 17877.917295.2 582.7
Original Size (meg) 50145.944692.7 5453.2
So, my questions are:
1) should I be doing hardware compression or software compression? (I
can spare the cycles, and I believe the FAQ says that software
compression allows the planner to do it's best job)
2) Why did the planner choose level 2 for those partitions, but then
amanda try and do a level 0 (thereby running itself out of tape)?
3) I am using 20/40 gb tapes; I got just
Output Size (meg) 17877.917295.2 582.7
on TWO tapes this last run -- any ideas why or what I can do to
improve this?
This is already lengthy; I can provide full config files and
logs as necessary.
Thanks for your time and assistance.
Lynette Bellini
Systems Administrator
University of Minnesota
