Re: 4.x can't read 5.x dump?
On Dec 3, 2004, at 21:03, Adrian Wontroba wrote: On Fri, Dec 03, 2004 at 02:36:09PM +, Ceri Davies wrote: Should I expect a dump taken from 4.X to be restorable on 5.X though? Yes. Phew. I didn't even think about the possibility of dump not being forwards compatible (8-( Maybe a note should be added to the dump(8) man page in the 5.x and HEAD branches noting this fact? At the very least an entry in UPDATING perhaps? It seems obvious once you think about it, but things tend to just work with compatibility with FreeBSD that you take it for granted. : ___ [EMAIL PROTECTED] mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: 4.x can't read 5.x dump?
On Wed, Dec 01, 2004 at 09:40:53PM -0500, Ken Smith wrote: On Thu, Dec 02, 2004 at 10:48:43AM +1000, Joel Hatton wrote: I'm backing up a 5.x machine at the moment with this command: dump -0Lau -b128 -f - /var | gzip -2 | ssh FreeBSD4 dd of=aacd0s1f.gz After the dump finishes, I try to read the file on the 4.x destination: # gzip -dc aacd0s1a.gz | restore -ivf - Verify tape and initialize maps Tape is not a dump tape I can scp the file back to the 5.x machine and it loads just fine, so what gives? This type of failure is somewhat scary for me right now, given that I may have to restore files to another destination that may not be 5.x based. This is, unfortunately, something that you should not expect to work for any *nix variant. The dump mechanism of creating backups creates output that has intimate knowledge of the filesystem so that it can recreate that filesystem exactly as it exists on the disk. As the filesystem itself evolves (has features added to it) the dump programs need to have their data structures change to accomodate the extra information that is now needed. For example if in 4.X there were no ACLs but 5.X added ACLs then the dump program's data structures would need to be changed so it had the ability to store the ACL information in the output, and the restore program would need to be modified to look for that and do the right thing if it finds ACL information. If there had been no filesystem changes between 4.X and 5.X then there would not be any need to change dump, and what you are trying to do would work. But there were filesystem changes between 4.X and 5.X, so dump was changed, and now there is extra stuff in the dump images that the 4.X version of restore doesn't understand. This same general principle holds for all OS's, not just FreeBSD. Should I expect a dump taken from 4.X to be restorable on 5.X though? (I do). Ceri -- Only two things are infinite, the universe and human stupidity, and I'm not sure about the former.-- Einstein (attrib.) pgpktWmj3aUyv.pgp Description: PGP signature
Re: 4.x can't read 5.x dump?
On Fri, Dec 03, 2004 at 02:24:15PM +, Ceri Davies wrote: Should I expect a dump taken from 4.X to be restorable on 5.X though? (I do). Yes. -- Ken Smith - From there to here, from here to | [EMAIL PROTECTED] there, funny things are everywhere. | - Theodore Geisel | ___ [EMAIL PROTECTED] mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: 4.x can't read 5.x dump?
On Fri, Dec 03, 2004 at 09:27:07AM -0500, Ken Smith wrote: On Fri, Dec 03, 2004 at 02:24:15PM +, Ceri Davies wrote: Should I expect a dump taken from 4.X to be restorable on 5.X though? (I do). Yes. Phew. Ceri -- Only two things are infinite, the universe and human stupidity, and I'm not sure about the former.-- Einstein (attrib.) pgpAZ96A0iaJk.pgp Description: PGP signature
Re: 4.x can't read 5.x dump?
On Fri, Dec 03, 2004 at 02:36:09PM +, Ceri Davies wrote: Should I expect a dump taken from 4.X to be restorable on 5.X though? Yes. Phew. I didn't even think about the possibility of dump not being forwards compatible (8-( In passing, you may find the buffer port useful. I spent a while a couple of years ago experimenting with settings, and find this sort of thing sufficiently fast: /sbin/dump -L -0 -u -C 32 -b 32 -f - /home | /usr/local/bin/buffer -s 32k -m 16m -t | gzip | /usr/local/bin/buffer -s 32k -m 16m -t new/home.dump.gz That is, big buffers and block sizes for dump, with a 16 MB buffer of either side of gzip to absorb some of the delays in compression or writing the dump file. Sorry, no timings available at the moment. Note: * Remove the -L under Release 4 * When the above line is executed, the script has changed its CWD to the dump directory, usually on an NFS mount. * When dumping to the same machine, I include -h 0 and make the dump directory (and a slew of other recoverable by other ways ones) nodump with chflags nodump. -- Adrian Wontroba ___ [EMAIL PROTECTED] mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: 4.x can't read 5.x dump?
At 2004-12-02 02:40:53+, Ken Smith writes: On Thu, Dec 02, 2004 at 10:48:43AM +1000, Joel Hatton wrote: I'm backing up a 5.x machine at the moment with this command: dump -0Lau -b128 -f - /var | gzip -2 | ssh FreeBSD4 dd of=aacd0s1f.gz After the dump finishes, I try to read the file on the 4.x destination: # gzip -dc aacd0s1a.gz | restore -ivf - Verify tape and initialize maps Tape is not a dump tape I can scp the file back to the 5.x machine and it loads just fine, so what gives? This type of failure is somewhat scary for me right now, given that I may have to restore files to another destination that may not be 5.x based. This is, unfortunately, something that you should not expect to work for any *nix variant. There's no theoretical reason why the formats used by dump and restore shouldn't be forward and backward compatible, allowing an older restore (to an older filesystem type) to pick out the parts of the dump which make sense to it while ignoring parts which it doesn't understand. But they aren't, so it can't, so you're out of luck. [In theory, the filesystem could package itself, so an old restore binary running on a newer filesystem and given a newer dump would DTRT]. Bikeshed, bikeshed. Nick B ___ [EMAIL PROTECTED] mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to [EMAIL PROTECTED]
RE: 4.x can't read 5.x dump?
There's no theoretical reason why the formats used by dump and restore shouldn't be forward and backward compatible, allowing an older restore (to an older filesystem type) to pick out the parts of the dump which make sense to it while ignoring parts which it doesn't understand. But they aren't, so it can't, so you're out of luck. This is a pretty interesting issue that I didn't realize. I've regularly restored dumps from a Solaris 8 machine to my FreeBSD 4.x machines. (We had data volumes on some old Solaris machines that I replaced with FreeBSD.) I guess FreeBSD and Solaris volumes are similar enough that the restore just worked. Jaime Bozza ___ [EMAIL PROTECTED] mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: 4.x can't read 5.x dump?
On Thursday 02 December 2004 05:55, Nick Barnes wrote: There's no theoretical reason why the formats used by dump and restore shouldn't be forward and backward compatible, allowing an older restore (to an older filesystem type) to pick out the parts of the dump which make sense to it while ignoring parts which it doesn't understand. There may be security reasons, though. If you've locked down a filesystem through the extensive use of ACLs, then you wouldn't want to dump/restore it to another system that doesn't support those ACLs and expect your security framework to still be intact - at least not without *lots* of warnings. -- Kirk Strauser pgpPgmSMhN3Us.pgp Description: PGP signature
4.x can't read 5.x dump?
Hi, I'm backing up a 5.x machine at the moment with this command: dump -0Lau -b128 -f - /var | gzip -2 | ssh FreeBSD4 dd of=aacd0s1f.gz After the dump finishes, I try to read the file on the 4.x destination: # gzip -dc aacd0s1a.gz | restore -ivf - Verify tape and initialize maps Tape is not a dump tape I can scp the file back to the 5.x machine and it loads just fine, so what gives? This type of failure is somewhat scary for me right now, given that I may have to restore files to another destination that may not be 5.x based. thanks, -- Joel Hatton -- Security Analyst| Hotline: +61 7 3365 4417 AusCERT - Australia's national CERT | Fax: +61 7 3365 7031 The University of Queensland| WWW: www.auscert.org.au Qld 4072 Australia | Email: [EMAIL PROTECTED] ___ [EMAIL PROTECTED] mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to [EMAIL PROTECTED]
4.x can't read 5.x dump?
Hi, I'm backing up a 5.x machine at the moment with this command: dump -0Lau -b128 -f - /var | gzip -2 | ssh FreeBSD4 dd of=aacd0s1f.gz After the dump finishes, I try to read the file on the 4.x destination: # gzip -dc aacd0s1a.gz | restore -ivf - Verify tape and initialize maps Tape is not a dump tape I can scp the file back to the 5.x machine and it loads just fine, so what gives? This type of failure is somewhat scary for me right now, given that I may have to restore files to another destination that may not be 5.x based. thanks, -- Joel Hatton -- Security Analyst| Hotline: +61 7 3365 4417 AusCERT - Australia's national CERT | Fax: +61 7 3365 7031 The University of Queensland| WWW: www.auscert.org.au Qld 4072 Australia | Email: [EMAIL PROTECTED] ___ [EMAIL PROTECTED] mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: 4.x can't read 5.x dump?
On Thu, 2 Dec 2004 11:19, [EMAIL PROTECTED] wrote: I can scp the file back to the 5.x machine and it loads just fine, so what gives? This type of failure is somewhat scary for me right now, given that I may have to restore files to another destination that may not be 5.x based. Is the 5.x dump a UFS 1 or 2 FS? If it's 2 then it's not really suprising since 4.x can't read it. You could quite proabably recompile restore with some kludgy support for UFS2 though. -- Daniel O'Connor software and network engineer for Genesis Software - http://www.gsoft.com.au The nice thing about standards is that there are so many of them to choose from. -- Andrew Tanenbaum GPG Fingerprint - 5596 B766 97C0 0E94 4347 295E E593 DC20 7B3F CE8C pgpwcvW02daU6.pgp Description: PGP signature
Re: 4.x can't read 5.x dump?
On Thu, Dec 02, 2004 at 10:48:43AM +1000, Joel Hatton wrote: I'm backing up a 5.x machine at the moment with this command: dump -0Lau -b128 -f - /var | gzip -2 | ssh FreeBSD4 dd of=aacd0s1f.gz After the dump finishes, I try to read the file on the 4.x destination: # gzip -dc aacd0s1a.gz | restore -ivf - Verify tape and initialize maps Tape is not a dump tape I can scp the file back to the 5.x machine and it loads just fine, so what gives? This type of failure is somewhat scary for me right now, given that I may have to restore files to another destination that may not be 5.x based. This is, unfortunately, something that you should not expect to work for any *nix variant. The dump mechanism of creating backups creates output that has intimate knowledge of the filesystem so that it can recreate that filesystem exactly as it exists on the disk. As the filesystem itself evolves (has features added to it) the dump programs need to have their data structures change to accomodate the extra information that is now needed. For example if in 4.X there were no ACLs but 5.X added ACLs then the dump program's data structures would need to be changed so it had the ability to store the ACL information in the output, and the restore program would need to be modified to look for that and do the right thing if it finds ACL information. If there had been no filesystem changes between 4.X and 5.X then there would not be any need to change dump, and what you are trying to do would work. But there were filesystem changes between 4.X and 5.X, so dump was changed, and now there is extra stuff in the dump images that the 4.X version of restore doesn't understand. This same general principle holds for all OS's, not just FreeBSD. If you need to be able to restore stuff from a 5.X machine on a variety of different platforms (4.X, or some other *nix) then don't use dump to create the images, use something like Gnu tar which can be made to understand the concept of incremental backups. You do however run the risk of not being able to do a perfect restore of a filesystem if you use some of the more advanced filesystem features like immutable files or ACLs - the tar mechanisms of doing backups may not be able to record that extra information. -- Ken Smith - From there to here, from here to | [EMAIL PROTECTED] there, funny things are everywhere. | - Theodore Geisel | ___ [EMAIL PROTECTED] mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: 4.x can't read 5.x dump?
Thanks Ken (and thanks to Paul, who replied with similar info), I can appreciate this - I shouldn't have been, but I guess I was just caught out by the change in the filesystem type and didn't realise that dump depended so heavily on it. I can work around, so all is well. Sorry about the double posting btw, I didn't expect my first one to make it to the list. cheers, -- Joel Hatton -- Security Analyst| Hotline: +61 7 3365 4417 AusCERT - Australia's national CERT | Fax: +61 7 3365 7031 The University of Queensland| WWW: www.auscert.org.au Qld 4072 Australia | Email: [EMAIL PROTECTED] On Thu, Dec 02, 2004 at 10:48:43AM +1000, Joel Hatton wrote: I'm backing up a 5.x machine at the moment with this command: dump -0Lau -b128 -f - /var | gzip -2 | ssh FreeBSD4 dd of=aacd0s1f.gz After the dump finishes, I try to read the file on the 4.x destination: # gzip -dc aacd0s1a.gz | restore -ivf - Verify tape and initialize maps Tape is not a dump tape I can scp the file back to the 5.x machine and it loads just fine, so what gives? This type of failure is somewhat scary for me right now, given that I may have to restore files to another destination that may not be 5.x based. This is, unfortunately, something that you should not expect to work for any *nix variant. The dump mechanism of creating backups creates output that has intimate knowledge of the filesystem so that it can recreate that filesystem exactly as it exists on the disk. As the filesystem itself evolves (has features added to it) the dump programs need to have their data structures change to accomodate the extra information that is now needed. For example if in 4.X there were no ACLs but 5.X added ACLs then the dump program's data structures would need to be changed so it had the ability to store the ACL information in the output, and the restore program would need to be modified to look for that and do the right thing if it finds ACL information. If there had been no filesystem changes between 4.X and 5.X then there would not be any need to change dump, and what you are trying to do would work. But there were filesystem changes between 4.X and 5.X, so dump was changed, and now there is extra stuff in the dump images that the 4.X version of restore doesn't understand. This same general principle holds for all OS's, not just FreeBSD. If you need to be able to restore stuff from a 5.X machine on a variety of different platforms (4.X, or some other *nix) then don't use dump to create the images, use something like Gnu tar which can be made to understand the concept of incremental backups. You do however run the risk of not being able to do a perfect restore of a filesystem if you use some of the more advanced filesystem features like immutable files or ACLs - the tar mechanisms of doing backups may not be able to record that extra information. -- Ken Smith - From there to here, from here to | [EMAIL PROTECTED] there, funny things are everywhere. | - Theodore Geisel | ___ [EMAIL PROTECTED] mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to [EMAIL PROTECTED]
