Re: If most recent backup is not level 0, recovery fails to bring back all files when directories have been renamed
Paul Bijnens schrieb: >> Why not implement some check-routine into Amanda that checks for the >> installed tar-release and gives a WARNING if a well-known problematic >> release is found? > > That would give a lot of false positives. > > RedHat for example backports important bugfixes to earlier versions > of programs. And while the "tar --version" here says "1.14" > Redhad has its own numbering scheme found by "rpm -q tar", resulting > in "tar-1.14-9.RHEL4", on my machine. And that version has all the > known critical bugs fixed, included the one above. Ok, then it has to get pointed out more clearly at installation-time that the user has to check for a proper release by himself or something. Users don't check the docs for problems they don't have/see, most of them even don't check the docs if they actually have/see a problem ;-) Stefan.
Re: If most recent backup is not level 0, recovery fails to bring back all files when directories have been renamed
On 2006-03-16 18:37, Stefan G. Weichinger wrote: Dave Ewart schrieb: On Wednesday, 08.03.2006 at 10:27 -0500, Ian Turner wrote: Please try upgrading your tar and check if you still see this behaviour. OK, you've hit the nail on the head: Using the existing system tar: $ tar --version tar (GNU tar) 1.14 And using a rebuilt version from a more recent source: $ ~/src/tar-1.15.1/src/tar --version tar (GNU tar) 1.15.1 A new tar is indeed the answer it seems. Suggestion related to the tar-issues: Why not implement some check-routine into Amanda that checks for the installed tar-release and gives a WARNING if a well-known problematic release is found? That would give a lot of false positives. RedHat for example backports important bugfixes to earlier versions of programs. And while the "tar --version" here says "1.14" Redhad has its own numbering scheme found by "rpm -q tar", resulting in "tar-1.14-9.RHEL4", on my machine. And that version has all the known critical bugs fixed, included the one above. -- Paul Bijnens, xplanation Technology ServicesTel +32 16 397.511 Technologielaan 21 bus 2, B-3001 Leuven, BELGIUMFax +32 16 397.512 http://www.xplanation.com/ email: [EMAIL PROTECTED] *** * I think I've got the hang of it now: exit, ^D, ^C, ^\, ^Z, ^Q, ^^, * * F6, quit, ZZ, :q, :q!, M-Z, ^X^C, logoff, logout, close, bye, /bye, * * stop, end, F3, ~., ^]c, +++ ATH, disconnect, halt, abort, hangup, * * PF4, F20, ^X^X, :D::D, KJOB, F14-f-e, F8-e, kill -1 $$, shutdown, * * init 0, kill -9 1, Alt-F4, Ctrl-Alt-Del, AltGr-NumLock, Stop-A, ... * * ... "Are you sure?" ... YES ... Phew ... I'm out * ***
Re: If most recent backup is not level 0, recovery fails to bring back all files when directories have been renamed
Dave Ewart schrieb: > On Wednesday, 08.03.2006 at 10:27 -0500, Ian Turner wrote: > >> Please >> try upgrading your tar and check if you still see this behaviour. > > OK, you've hit the nail on the head: > > Using the existing system tar: > > $ tar --version > tar (GNU tar) 1.14 > And using a rebuilt version from a more recent source: > > $ ~/src/tar-1.15.1/src/tar --version > tar (GNU tar) 1.15.1 > A new tar is indeed the answer it seems. Suggestion related to the tar-issues: Why not implement some check-routine into Amanda that checks for the installed tar-release and gives a WARNING if a well-known problematic release is found? For the wishlist, I assume ;-) Stefan
Re: If most recent backup is not level 0, recovery fails to bring back all files when directories have been renamed
On Wednesday, 08.03.2006 at 12:05 -0500, Jon LaBadie wrote: > So, rather than use "listed-incremental" to recognize > it is a renamed directory, gnutar's behavior is to > consider then entire directory tree "changed" and > back up the entire tree. > > Reading this thread reminded me of another one where > the poster was complaining about this "feature" of > gnutar. That poster has a lot of rotating log dirs > that were quite large. They were rotated by renaming > them each night (log.2 becomes log.3, log.1 -> log.2 etc.) > Their complaint was that the data in the directory WAS > being backed up each night though it was unchanging. > > Be nice if gnutar could deal with both needs. Indeed, that very issue had occurred to me too :-) However, directory renames are fairly rare and IMO if there is an 'error' it should be "back up too much but making sure it's complete" rather than "keep the backup size small but occasionally miss something important". Of course, supporting both Would Be Nice, as you say... Dave. -- Dave Ewart [EMAIL PROTECTED] Computing Manager, Cancer Epidemiology Unit Cancer Research UK / Oxford University PGP: CC70 1883 BD92 E665 B840 118B 6E94 2CFD 694D E370 Get key from http://www.ceu.ox.ac.uk/~davee/davee-ceu-ox-ac-uk.asc N 51.7518, W 1.2016 signature.asc Description: Digital signature
Re: If most recent backup is not level 0, recovery fails to bring back all files when directories have been renamed
--On March 8, 2006 11:34:16 AM + Dave Ewart <[EMAIL PROTECTED]> wrote: Thoughts/opinions here? BTW the AMANDA server runs Debian/Woody (2.4.2p2-4) and the client being backed up above runs Debian/Sarge AMD64 (2.4.4p3-3). It could very well be b -- if it is it's not amanda, it's tar. The backup program is responsible exclusively for what does and does not get backed up. Amanda just communicates a level/last backup date/file list to use. Depending on the dump/backup program. I know that a new tar version in sarge atleast before the security update taht just went out is broken anyway. It doesn't create valid archives a lot of the time, you'd be better off installing a backported tar from unstable, or rebuilding the one in oldstable and using that. Google around for invalid base64 or obsolescent base64 header skipping to next archive (i think that's right) if you haven't seen the error message yet. Cheers, Dave. -- Dave Ewart [EMAIL PROTECTED] Computing Manager, Cancer Epidemiology Unit Cancer Research UK / Oxford University PGP: CC70 1883 BD92 E665 B840 118B 6E94 2CFD 694D E370 Get key from http://www.ceu.ox.ac.uk/~davee/davee-ceu-ox-ac-uk.asc N 51.7518, W 1.2016 -- "Genius might be described as a supreme capacity for getting its possessors into trouble of all kinds." -- Samuel Butler
Re: If most recent backup is not level 0, recovery fails to bring back all files when directories have been renamed
On Wed, Mar 08, 2006 at 04:21:42PM +, Dave Ewart wrote: > On Wednesday, 08.03.2006 at 10:27 -0500, Ian Turner wrote: > > > This does not appear to happen with CVS tar in listed-incremental mode. > > Please > > try upgrading your tar and check if you still see this behaviour. > > > > OK, you've hit the nail on the head: > [snip] > > > A new tar is indeed the answer it seems. So, rather than use "listed-incremental" to recognize it is a renamed directory, gnutar's behavior is to consider then entire directory tree "changed" and back up the entire tree. Reading this thread reminded me of another one where the poster was complaining about this "feature" of gnutar. That poster has a lot of rotating log dirs that were quite large. They were rotated by renaming them each night (log.2 becomes log.3, log.1 -> log.2 etc.) Their complaint was that the data in the directory WAS being backed up each night though it was unchanging. Be nice if gnutar could deal with both needs. -- Jon H. LaBadie [EMAIL PROTECTED] JG Computing 4455 Province Line Road(609) 252-0159 Princeton, NJ 08540-4322 (609) 683-7220 (fax)
Re: If most recent backup is not level 0, recovery fails to bring back all files when directories have been renamed
On Wednesday, 08.03.2006 at 10:27 -0500, Ian Turner wrote: > On Wednesday 08 March 2006 06:34, Dave Ewart wrote: > > In summary: when a directory is *renamed* the files underneath it are > > not considered to have changed > > This does not appear to happen with CVS tar in listed-incremental mode. > Please > try upgrading your tar and check if you still see this behaviour. > > For reference, this is the test I performed: > > $ # Build CVS tar, chdir to src/. > $ mkdir -p test/foo > $ touch test/foo/bar > $ ./tar -cvf test1.tar --listed-incremental test1.incr test > ./tar: test/foo: Directory is new > test/ > test/foo/ > test/foo/bar > $ mv test/foo test/baz > $ ./tar -cvf test2.tar --listed-incremental test1.incr test > ./tar: test/baz: Directory is new > test/ > test/baz/ > test/baz/bar OK, you've hit the nail on the head: Using the existing system tar: $ tar --version tar (GNU tar) 1.14 $ find . ./blah ./blah/a ./blah/a/foo ./blah/a/bar $ tar -cvf test1.tar --listed-incremental test1.incr blah tar: blah/a: Directory is new blah/ blah/a/ blah/a/bar blah/a/foo $ mv blah/a blah/b $ tar -cvf test1.tar --listed-incremental test1.incr blah tar: blah/b: Directory is new blah/ blah/b/ And using a rebuilt version from a more recent source: $ ~/src/tar-1.15.1/src/tar --version tar (GNU tar) 1.15.1 $ ~/src/tar-1.15.1/src/tar -cvf test1.tar --listed-incremental test1.incr blah /home/davee/src/tar-1.15.1/src/tar: blah/a: Directory is new blah/ blah/a/ blah/a/bar blah/a/foo $ mv blah/a blah/b $ ~/src/tar-1.15.1/src/tar -cvf test1.tar --listed-incremental test1.incr blah /home/davee/src/tar-1.15.1/src/tar: blah/b: Directory is new blah/ blah/b/ blah/b/bar blah/b/foo A new tar is indeed the answer it seems. Many thanks, people. Dave. -- Dave Ewart [EMAIL PROTECTED] Computing Manager, Cancer Epidemiology Unit Cancer Research UK / Oxford University PGP: CC70 1883 BD92 E665 B840 118B 6E94 2CFD 694D E370 Get key from http://www.ceu.ox.ac.uk/~davee/davee-ceu-ox-ac-uk.asc N 51.7518, W 1.2016 signature.asc Description: Digital signature
Re: If most recent backup is not level 0, recovery fails to bring back all files when directories have been renamed
On Wednesday, 08.03.2006 at 10:02 -0500, Gene Heskett wrote: > >This package of 'tar' is the current default in Debian/Sarge and is > >updated with various security issues without changing the version > >number, so it's not entirely clear what version is really there under > >the hood. > > Which is exactly why I get this stuff (both tar and amanda) from the > src and build my own. It Just Works(TM). :) Well, that works both ways of course. The latest source might introduce new problems, and at least distro-issued packages have been preconfigured to put the files in 'all the right places', as per the distro's usual specifications ... I'll investigate what Ian has posted about listed-incremental... Dave. -- Dave Ewart [EMAIL PROTECTED] Computing Manager, Cancer Epidemiology Unit Cancer Research UK / Oxford University PGP: CC70 1883 BD92 E665 B840 118B 6E94 2CFD 694D E370 Get key from http://www.ceu.ox.ac.uk/~davee/davee-ceu-ox-ac-uk.asc N 51.7518, W 1.2016 signature.asc Description: Digital signature
Re: If most recent backup is not level 0, recovery fails to bring back all files when directories have been renamed
On Wed, Mar 08, 2006 at 10:31:44AM -0500, Gene Heskett enlightened us: > >I hope you build your own RPMs, since I know you use an RPM based > > system ;-) > > Nope, I use checkinstall for some stuff, but this isn't being done for > these. And they are pinned in yum.conf. Yeah, my systems borked. > Funny thing is, it works fine for everything I want to do, which at > times can best be described as 'an eclectic bunch' of apps. > I know we're drifting a bit off-topic here, but usually what I do for non-mission-critical machines (such as a home workstation) is grab stuff from the latest fedora development repository and rebuild them on my system. That's what I did for tar, and what I do for amanda (even on production systems I do that). Just a suggestion :-) Matt -- Matt Hyclak Department of Mathematics Department of Social Work Ohio University (740) 593-1263
Re: If most recent backup is not level 0, recovery fails to bring back all files when directories have been renamed
On Wednesday 08 March 2006 10:17, Matt Hyclak wrote: >On Wed, Mar 08, 2006 at 10:02:43AM -0500, Gene Heskett enlightened us: >> >On Wednesday, 08.03.2006 at 07:35 -0500, Gene Heskett wrote: >> >> >On the client being backed-up: >> >> > >> >> >$ tar --version >> >> > >> >> >tar (GNU tar) 1.14 >> >> >> >> We believe this version of tar is borked. Please get the older >> >> 1.13-19, 1.13-25, or the newer 1.15-1. For whatever reason, 1.14 >> >> was only visible on gnu.org for about 5 or 6 weeks, being >> >> replaced with 1.15-1, which for gnu.org, considering the speed >> >> they normally run at, is instantainious. I've been running >> >> 1.15-1 since about a week after it became available without >> >> problems. >> > >> >Interesting. Do you think that the behaviour I'm seeing is solely >> > as a result of 'tar' here? >> > >> >Is the behaviour I describe something that you believe *should* >> > *not* happen with AMANDA when using 'tar'? >> >> I do not know this for a fact. ISTR the file headers it made >> weren't correct and recovery failures were the result. Possibly >> someone else can elaborate here? > >It created proper archives, however it failed extracting archives that > had sparse files in them properly. RedHat still uses this version in > RHEL4, but has backported the fix for this problem from 1.15.1. > >> >This package of 'tar' is the current default in Debian/Sarge and is >> >updated with various security issues without changing the version >> >number, so it's not entirely clear what version is really there >> > under the hood. >> >> Which is exactly why I get this stuff (both tar and amanda) from the >> src and build my own. It Just Works(TM). :) > >I hope you build your own RPMs, since I know you use an RPM based > system ;-) Nope, I use checkinstall for some stuff, but this isn't being done for these. And they are pinned in yum.conf. Yeah, my systems borked. Funny thing is, it works fine for everything I want to do, which at times can best be described as 'an eclectic bunch' of apps. >Matt -- Cheers, Gene People having trouble with vz bouncing email to me should add the word 'online' between the 'verizon', and the dot which bypasses vz's stupid bounce rules. I do use spamassassin too. :-) Yahoo.com and AOL/TW attorneys please note, additions to the above message by Gene Heskett are: Copyright 2006 by Maurice Eugene Heskett, all rights reserved.
Re: If most recent backup is not level 0, recovery fails to bring back all files when directories have been renamed
On Wednesday 08 March 2006 06:34, Dave Ewart wrote: > In summary: when a directory is *renamed* the files underneath it are > not considered to have changed This does not appear to happen with CVS tar in listed-incremental mode. Please try upgrading your tar and check if you still see this behaviour. For reference, this is the test I performed: $ # Build CVS tar, chdir to src/. $ mkdir -p test/foo $ touch test/foo/bar $ ./tar -cvf test1.tar --listed-incremental test1.incr test ./tar: test/foo: Directory is new test/ test/foo/ test/foo/bar $ mv test/foo test/baz $ ./tar -cvf test2.tar --listed-incremental test1.incr test ./tar: test/baz: Directory is new test/ test/baz/ test/baz/bar Cheers, --Ian -- Forums for Amanda discussion: http://forums.zmanda.com/
Re: If most recent backup is not level 0, recovery fails to bring back all files when directories have been renamed
On Wed, Mar 08, 2006 at 10:02:43AM -0500, Gene Heskett enlightened us: > >On Wednesday, 08.03.2006 at 07:35 -0500, Gene Heskett wrote: > >> >On the client being backed-up: > >> > > >> >$ tar --version > >> > > >> >tar (GNU tar) 1.14 > >> > >> We believe this version of tar is borked. Please get the older > >> 1.13-19, 1.13-25, or the newer 1.15-1. For whatever reason, 1.14 > >> was only visible on gnu.org for about 5 or 6 weeks, being replaced > >> with 1.15-1, which for gnu.org, considering the speed they normally > >> run at, is instantainious. I've been running 1.15-1 since about a > >> week after it became available without problems. > > > >Interesting. Do you think that the behaviour I'm seeing is solely as > > a result of 'tar' here? > > > >Is the behaviour I describe something that you believe *should* *not* > >happen with AMANDA when using 'tar'? > > > I do not know this for a fact. ISTR the file headers it made weren't > correct and recovery failures were the result. Possibly someone else > can elaborate here? > It created proper archives, however it failed extracting archives that had sparse files in them properly. RedHat still uses this version in RHEL4, but has backported the fix for this problem from 1.15.1. > >This package of 'tar' is the current default in Debian/Sarge and is > >updated with various security issues without changing the version > >number, so it's not entirely clear what version is really there under > >the hood. > > Which is exactly why I get this stuff (both tar and amanda) from the src > and build my own. It Just Works(TM). :) > I hope you build your own RPMs, since I know you use an RPM based system ;-) Matt -- Matt Hyclak Department of Mathematics Department of Social Work Ohio University (740) 593-1263
Re: If most recent backup is not level 0, recovery fails to bring back all files when directories have been renamed
On Wednesday 08 March 2006 08:43, Dave Ewart wrote: >On Wednesday, 08.03.2006 at 07:35 -0500, Gene Heskett wrote: >> >On the client being backed-up: >> > >> >$ tar --version >> > >> >tar (GNU tar) 1.14 >> >> We believe this version of tar is borked. Please get the older >> 1.13-19, 1.13-25, or the newer 1.15-1. For whatever reason, 1.14 >> was only visible on gnu.org for about 5 or 6 weeks, being replaced >> with 1.15-1, which for gnu.org, considering the speed they normally >> run at, is instantainious. I've been running 1.15-1 since about a >> week after it became available without problems. > >Interesting. Do you think that the behaviour I'm seeing is solely as > a result of 'tar' here? > >Is the behaviour I describe something that you believe *should* *not* >happen with AMANDA when using 'tar'? > I do not know this for a fact. ISTR the file headers it made weren't correct and recovery failures were the result. Possibly someone else can elaborate here? >This package of 'tar' is the current default in Debian/Sarge and is >updated with various security issues without changing the version >number, so it's not entirely clear what version is really there under >the hood. Which is exactly why I get this stuff (both tar and amanda) from the src and build my own. It Just Works(TM). :) In this case I put the new version in /usr/local/bin & reconfigured & rebuilt amanda to look for it there, this so I could switch back if I needed to. But I never did need to. >Dave. -- Cheers, Gene People having trouble with vz bouncing email to me should add the word 'online' between the 'verizon', and the dot which bypasses vz's stupid bounce rules. I do use spamassassin too. :-) Yahoo.com and AOL/TW attorneys please note, additions to the above message by Gene Heskett are: Copyright 2006 by Maurice Eugene Heskett, all rights reserved.
Re: If most recent backup is not level 0, recovery fails to bring back all files when directories have been renamed
On Wednesday 08 March 2006 06:34, Dave Ewart wrote: > In summary: when a directory is *renamed* the files underneath it are > not considered to have changed Usually, I would say that this is a limitation of the UNIX filesystem. A directory's modification time can change anytime you add or remove a file in that directory. Obviously it doesn't make sense to re-backup every file in a hierarchy just because you created or removed one. However, in this case you are using gnutar, which creates listed incrementals (I assume you did not configure --without-gnutar-listdir) that can be used to detect the renaming of a directory. Therefore I have forwarded your complaint to the GNU tar folks. Cheers, --Ian -- Forums for Amanda discussion: http://forums.zmanda.com/
Re: If most recent backup is not level 0, recovery fails to bring back all files when directories have been renamed
On Wednesday, 08.03.2006 at 07:35 -0500, Gene Heskett wrote: > >On the client being backed-up: > > > >$ tar --version > > > >tar (GNU tar) 1.14 > > We believe this version of tar is borked. Please get the older > 1.13-19, 1.13-25, or the newer 1.15-1. For whatever reason, 1.14 was > only visible on gnu.org for about 5 or 6 weeks, being replaced with > 1.15-1, which for gnu.org, considering the speed they normally run at, > is instantainious. I've been running 1.15-1 since about a week after > it became available without problems. Interesting. Do you think that the behaviour I'm seeing is solely as a result of 'tar' here? Is the behaviour I describe something that you believe *should* *not* happen with AMANDA when using 'tar'? This package of 'tar' is the current default in Debian/Sarge and is updated with various security issues without changing the version number, so it's not entirely clear what version is really there under the hood. Dave. -- Dave Ewart [EMAIL PROTECTED] Computing Manager, Cancer Epidemiology Unit Cancer Research UK / Oxford University PGP: CC70 1883 BD92 E665 B840 118B 6E94 2CFD 694D E370 Get key from http://www.ceu.ox.ac.uk/~davee/davee-ceu-ox-ac-uk.asc N 51.7518, W 1.2016 signature.asc Description: Digital signature
Re: If most recent backup is not level 0, recovery fails to bring back all files when directories have been renamed
On Wednesday 08 March 2006 06:50, Dave Ewart wrote: >On Wednesday, 08.03.2006 at 11:34 +, Dave Ewart wrote: >> I've just been caught out by something which is either >> >> (a) a limitation of AMANDA and/or tar/dump; >> (b) a bug. > >A follow-up, prompted by a reply from Paul, indicating that I had > indeed forgotten a couple of rather important bits of information! > >- The filesystem being discussed is ext3 > >- AMANDA is using 'tar' (i.e. amanda.conf states: program "GNUTAR") > >On the client being backed-up: > >$ tar --version > >tar (GNU tar) 1.14 We believe this version of tar is borked. Please get the older 1.13-19, 1.13-25, or the newer 1.15-1. For whatever reason, 1.14 was only visible on gnu.org for about 5 or 6 weeks, being replaced with 1.15-1, which for gnu.org, considering the speed they normally run at, is instantainious. I've been running 1.15-1 since about a week after it became available without problems. >Copyright (C) 2004 Free Software Foundation, Inc. >... > >Dave. -- Cheers, Gene People having trouble with vz bouncing email to me should add the word 'online' between the 'verizon', and the dot which bypasses vz's stupid bounce rules. I do use spamassassin too. :-) Yahoo.com and AOL/TW attorneys please note, additions to the above message by Gene Heskett are: Copyright 2006 by Maurice Eugene Heskett, all rights reserved.
Re: If most recent backup is not level 0, recovery fails to bring back all files when directories have been renamed
On Wednesday, 08.03.2006 at 11:34 +, Dave Ewart wrote: > I've just been caught out by something which is either > > (a) a limitation of AMANDA and/or tar/dump; > (b) a bug. A follow-up, prompted by a reply from Paul, indicating that I had indeed forgotten a couple of rather important bits of information! - The filesystem being discussed is ext3 - AMANDA is using 'tar' (i.e. amanda.conf states: program "GNUTAR") On the client being backed-up: $ tar --version tar (GNU tar) 1.14 Copyright (C) 2004 Free Software Foundation, Inc. ... Dave. -- Dave Ewart [EMAIL PROTECTED] Computing Manager, Cancer Epidemiology Unit Cancer Research UK / Oxford University PGP: CC70 1883 BD92 E665 B840 118B 6E94 2CFD 694D E370 Get key from http://www.ceu.ox.ac.uk/~davee/davee-ceu-ox-ac-uk.asc N 51.7518, W 1.2016 signature.asc Description: Digital signature
If most recent backup is not level 0, recovery fails to bring back all files when directories have been renamed
I've just been caught out by something which is either (a) a limitation of AMANDA and/or tar/dump; (b) a bug. In summary: when a directory is *renamed* the files underneath it are not considered to have changed: this means that a non-level-0 backup following the directory rename only backs up the new directory entry, not the files inside the directory. This means that, if the disk fails following such a non-level-0 backup and requires a recovery, the full recovery will *not* recover the files inside the directory: the new directory will be recreated by amrecover, but will be empty. Let me demonstrate by means of a simple example: On Friday 03.03.2006, I do the following: $ ls /home/davee/stata/w total 24 -rw-r--r-- 1 davee users68 2006-02-28 16:25 Makefile -rw-r--r-- 1 davee users 479 2006-03-03 15:44 w.csv -rw-r--r-- 1 davee users 224 2006-01-19 19:21 w.do -rw-r--r-- 1 davee users 11147 2006-03-03 15:44 w.log These are files last edited on 03.03.2006 and on this night a level 0 backup took place. The index for this directory contains the directory and all its files as expected: # zgrep 'davee/stata' 20060303_0.gz /davee/stata/ /davee/stata/w/ /davee/stata/w/Makefile /davee/stata/w/w.csv /davee/stata/w/w.do /davee/stata/w/w.log On Monday 06.03.2006, no further changes are made to the files and a level 1 backup tapes place that evening. The index for that backup includes the directory entry as expected, but none of the files: # zgrep 'davee/stata' 20060306_1.gz /davee/stata/ /davee/stata/w/ On Tuesday 07.03.2006, I rename the directory 'w' to 'w2', but make no changes to the files inside the directory. The file structure now looks like this: $ mv w w2 $ ls -l /home/davee/stata/w2/ total 24 -rw-r--r-- 1 davee users68 2006-02-28 16:25 Makefile -rw-r--r-- 1 davee users 479 2006-03-03 15:44 w.csv -rw-r--r-- 1 davee users 224 2006-01-19 19:21 w.do -rw-r--r-- 1 davee users 11147 2006-03-03 15:44 w.log On the evening of 07.03.2006, a level 2 backup takes place and the index contains the new directory entry: # zgrep 'davee/stata' 20060307_2.gz /davee/stata/ /davee/stata/w2/ The files in the directory are not changed which is presumably why the files are not included in this index. (Importantly, though, the directory entry for davee/stata/w has now gone: this will matter during recovery.) SO ... (and this is big issue here) ... if the disk on which these files are stored fails at this point, a recovery DOES NOT RECOVER ALL FILES. I guessing that what happens is that the recovery process (which prompts for the tapes from 20060303, 20060306 and 20060307 in turn) does the following: - restores the contents of the level 0 backup from 20060303 which would leave the filesystem as follows: $ ls /home/davee/stata/w total 24 -rw-r--r-- 1 davee users68 2006-02-28 16:25 Makefile -rw-r--r-- 1 davee users 479 2006-03-03 15:44 w.csv -rw-r--r-- 1 davee users 224 2006-01-19 19:21 w.do -rw-r--r-- 1 davee users 11147 2006-03-03 15:44 w.log - restores the contents of the level 1 backup from 20060306 which won't make any further changes to this part of the filesystem; - restores the contents of the level 2 backup from 20060307 which introduces an emtpy directory davee/stata/w2 and REMOVES davee/stata/w because it DOESN'T EXIST ANYMORE, leaving the filesystem as follows: $ ls /home/davee/stata drwxr-xr-x 2 davee users4096 2006-03-03 15:44 w2 $ ls /home/davee/stata/w2 (nothing) Clearly the solution in this case is to perform a separate recovery of the level 0 backup and manually put the files back in place. However, in the context of a massive recovery of many thousands of files following a disk failure, events such as the one described above (which is merely a small worked example) will not be noticed. The point of my illustration is that even though one has all the appropriate tapes for the recovery process, running an amrecover and feeding in the tapes one by one will NOT RESTORE ALL FILES, if both of the following circumstances are true: - the most recent backup prior to disk failure was not a level 0 backup; - directories were renamed since the last level 0 backup. Thoughts/opinions here? BTW the AMANDA server runs Debian/Woody (2.4.2p2-4) and the client being backed up above runs Debian/Sarge AMD64 (2.4.4p3-3). Cheers, Dave. -- Dave Ewart [EMAIL PROTECTED] Computing Manager, Cancer Epidemiology Unit Cancer Research UK / Oxford University PGP: CC70 1883 BD92 E665 B840 118B 6E94 2CFD 694D E370 Get key from http://www.ceu.ox.ac.uk/~davee/davee-ceu-ox-ac-uk.asc N 51.7518, W 1.2016 signature.asc Description: Digital signature
