Re: Dump questions
On Mon, Feb 22, 2010 at 03:10:01PM +, Matthew Seaman wrote:
-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1
On 22/02/2010 14:30, Jerry McAllister wrote:
No. In multi-user, files are still changing. The snapshot could
possibly be made between parts of a change - between different writes
to the file, so there could be some inconsistency. In practice this
is not a big problem, but, single user with filesystems unmounted is
still the most absolute way of making sure a filesystem is quiescent
during a dump.
Umm you don't *need* to go to single user to ensure a consistent
filesystem dump: unmounting the partition is sufficient, or remounting
it read-only.
True. But, the problem with that, as you follow with is that it can
produce a lot of fudd from parts of the running system that expect to
find that file system mountable and writable. Plus some filesystems
such as maybe /usr, etc may be needed for the multi-user system to
operate at all. So, you either don't dump them or go to single user
just for them or use -L and not worry about it, or whatever.
jerry
It's just that shutting the system down and rebooting to
single user mode can save you a deal of faffing about trying to kill
off any processes still using the filesystem, which would otherwise
block your ability to unmount it.
Note too, it's *reboot* into single user ('shutdown -r now', then press
4 at the boot menu) not *drop* into single user ('shutdown now') which
doesn't unmount filesystems for you, although it should kill almost all
processes.
Single user has it's own disadvantages: generally there's no network
configured, and with the root partition mounted read-only, you can't
update /etc/dumpdates.
Whenever you boot into single user, remember to run 'fsck -p' to ensure
filesystem integrity. I'm not sure what happens if you attempt to
dump'n'restore a dirty filesystem, but it's certainly going to have
unintended consequences if the filesystem is actually damaged rather
than just dirty.
Cheers,
Matthew
- --
Dr Matthew J Seaman MA, D.Phil. 7 Priory Courtyard
Flat 3
PGP: http://www.infracaninophile.co.uk/pgpkey Ramsgate
Kent, CT11 9PW
-BEGIN PGP SIGNATURE-
Version: GnuPG/MacGPG2 v2.0.14 (Darwin)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/
iEYEARECAAYFAkuCnkkACgkQ8Mjk52CukIyR+gCfX9rep9S9DQcIcRDqSoAptQX9
gMkAoIV/zhe4kRRlRN8fjn5+W7CS1csM
=6J2U
-END PGP SIGNATURE-
___
freebsd-questions@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-questions
To unsubscribe, send any mail to freebsd-questions-unsubscr...@freebsd.org
___
freebsd-questions@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-questions
To unsubscribe, send any mail to freebsd-questions-unsubscr...@freebsd.org
Re: Dump questions
On Mon, Feb 22, 2010 at 12:23:10PM +0800, Aiza wrote: Jerry McAllister wrote: On Sun, Feb 21, 2010 at 11:03:58AM +0100, Polytropon wrote: On Sun, 21 Feb 2010 11:42:50 +0800, Aiza aiz...@comclark.com wrote: 1. Using the -L flag to create a snapshot of the live running file system. ... Is this the limiting factor that forces a user to use (single user mode) for running dump? The snapshot, as far as the dump is concerned, essentially freezes the condition of the file system so that dump does not see any changes while dump is running. Without the snapshot, files could change or be deleted during the time it takes dump to finish. Dump starts by making its own list, by inode, of all the files to dump. Then it writes out, first the list, then the directories and finally the files and links to the media. If the files change between the time that list is made and things get written to the media, you will have an inaccurate representation of the file system. This can result in error messages if files it expects to be there are missing It can mean that a mangled image of a file is written in the dump. Doing a dump in Single User Mode means stopping activity on the system so there are fewer chances of the above happening. Using -L and doing a snapshot will not prevent a dump from being technically obsolete by the time it gets done, but it will mean that what gets written to media is internally consistent. The list it made will be exactly what is on the backup media and the files are all written completely as they were when the snapshot was taken with no mangling. 2. What is the worse that will happen if dump is run on live file system with out the -L flag? The index list that is written as part of the dump will not reflect what is on the dump media. It may claim a file is there, but it really is not. A file or some files are mangled because they are open and being modified by another process as the dump is reading them. The file may be either an inaccurate image or even completely unreadable. Restore is smart enough to skip over these problems if the file[s] you are looking to restore are not the ones mangled or deleted. But, you could get in to a situation of not being able to restore some things that you have on media. Can dump recognize this situation and issue an error message? I don't remember if dump puts out any useful diagnostic. I think it might tell you if it cannot file a file whose inode is in its list to write. But, it is restore that really notices and complains. If you have room, you can use restore to 'verify' a dump just by doing a restore of it to some extra space (maybe even to /dev/null, though I have never tried that one) and seeing if it makes any complaints. This, of course, is a long way to do this, but it might be valuable if it is essential for that dump to be completely readable in a later situation where the original is not longer available. But, in this situation, then making a -L dump (using a snapshot) is really important or even a single user, filesystem unmounted -L dump. 3. Can dump be told to only dump a particular directory tree? IE /var/log or /usr/port? dump only workes on filesystems/partitions. If you know you will want to make dumps on just that directory tree, that is a reason to make a separate partition/filesystem for it and mount it up. There is no reason that /var/log cannot be in its own partition/filesystem separate from /var and just mounted that way. Of course, you have to make sure that /var gets mounted before /var/log. But, that is not strange. Many people make a separate partition for /usr/home inside of /usr or a /var/db that is mounted inside of /var. Now, you can restore just a single file, group of files or a directory tree out of a dump. You do not have to restore the whole dump. So, you can make a dump of a '/var' filesystem if that is what you have and then if you need to restore just '/var/db' out of it, that is no problem. Just make sure you understand where you are putting it and how you specifiy it in the restore. But, if you just want a backup copy of a directory tree that is not its own partition/filesystem, you must use some other tool, such as tar or possibly rsync. jerry Thank you for the detail insight of how dump functions. Now one more question. Is the dump -L backup file made in a multiple-user-mode environment just as dependable as dump backups made in single-user-mode? No. In multi-user, files are still changing. The snapshot could possibly be made between parts of a change - between different writes to the file, so there could be some inconsistency. In practice this is not a big problem, but, single user with filesystems unmounted is still the most absolute way of making sure a filesystem is quiescent during a dump. jerry
Re: Dump questions
-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1
On 22/02/2010 14:30, Jerry McAllister wrote:
No. In multi-user, files are still changing. The snapshot could
possibly be made between parts of a change - between different writes
to the file, so there could be some inconsistency. In practice this
is not a big problem, but, single user with filesystems unmounted is
still the most absolute way of making sure a filesystem is quiescent
during a dump.
Umm you don't *need* to go to single user to ensure a consistent
filesystem dump: unmounting the partition is sufficient, or remounting
it read-only. It's just that shutting the system down and rebooting to
single user mode can save you a deal of faffing about trying to kill
off any processes still using the filesystem, which would otherwise
block your ability to unmount it.
Note too, it's *reboot* into single user ('shutdown -r now', then press
4 at the boot menu) not *drop* into single user ('shutdown now') which
doesn't unmount filesystems for you, although it should kill almost all
processes.
Single user has it's own disadvantages: generally there's no network
configured, and with the root partition mounted read-only, you can't
update /etc/dumpdates.
Whenever you boot into single user, remember to run 'fsck -p' to ensure
filesystem integrity. I'm not sure what happens if you attempt to
dump'n'restore a dirty filesystem, but it's certainly going to have
unintended consequences if the filesystem is actually damaged rather
than just dirty.
Cheers,
Matthew
- --
Dr Matthew J Seaman MA, D.Phil. 7 Priory Courtyard
Flat 3
PGP: http://www.infracaninophile.co.uk/pgpkey Ramsgate
Kent, CT11 9PW
-BEGIN PGP SIGNATURE-
Version: GnuPG/MacGPG2 v2.0.14 (Darwin)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/
iEYEARECAAYFAkuCnkkACgkQ8Mjk52CukIyR+gCfX9rep9S9DQcIcRDqSoAptQX9
gMkAoIV/zhe4kRRlRN8fjn5+W7CS1csM
=6J2U
-END PGP SIGNATURE-
___
freebsd-questions@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-questions
To unsubscribe, send any mail to freebsd-questions-unsubscr...@freebsd.org
Re: Dump questions
On Sun, Feb 21, 2010 at 03:45:51PM +0800, Aiza wrote: Dan Nelson wrote: In the last episode (Feb 21), Aiza said: 1. Using the -L flag to create a snapshot of the live running file system. Does this mean that a complete copy of the file system is written to .snap directory? No; that would be a copy. Snapshots only copy blocks as they are modified on the parent filesystem, so their size is determined by how much data is modified since the snapshot was created. So how does this interact with the dump process? Dump start reading and writing its dump file and as the live system changes the changes are written to the .snap and when dump completes it overwrites it dump with the changes from the .snap??? No. How does this process work in detail? Go back and read the good and quite complete description someone put in about how snapshotting works a while back in this thread. I think it was Matthew Seaman, but I don't remember for sure. jerry ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to freebsd-questions-unsubscr...@freebsd.org ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to freebsd-questions-unsubscr...@freebsd.org
Re: Dump questions
On Sun, 21 Feb 2010 11:42:50 +0800, Aiza aiz...@comclark.com wrote: 1. Using the -L flag to create a snapshot of the live running file system. Does this mean that a complete copy of the file system is written to .snap directory? No. The snapshot, quite incorrectly explained, is a saved delta between the file system on disk at a given state, to fixate further modifications (that are not included in the dump, of course). Is this the limiting factor that forces a user to use (single user mode) for running dump? Using SUM is for a feeling of comfort only. You can save the time needed for creating the snapshot by entering SUM - and, what's essential - unmount the partitions. This makes sure the underlying file systems aren't modified. 2. What is the worse that will happen if dump is run on live file system with out the -L flag? The dump could not be readable, which would imply that your backup is useless. Can dump recognize this situation and issue an error message? The dump program does what you tell it to do. It does not bother you with questions that you should have asked yourself already. :-) 3. Can dump be told to only dump a particular directory tree? IE /var/log or /usr/port? No. THe dump program operates on file systems. It does not have a concept of files and directories per se. If you plan to work with individual files and directories rather than partitions (file systems), check out tools like cpdup, rsync and the like. -- Polytropon Magdeburg, Germany Happy FreeBSD user since 4.0 Andra moi ennepe, Mousa, ... ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to freebsd-questions-unsubscr...@freebsd.org
Re: Dump questions
Polytropon wrote: On Sun, 21 Feb 2010 11:42:50 +0800, Aiza aiz...@comclark.com wrote: 1. Using the -L flag to create a snapshot of the live running file system. Does this mean that a complete copy of the file system is written to .snap directory? No. The snapshot, quite incorrectly explained, is a saved delta between the file system on disk at a given state, to fixate further modifications (that are not included in the dump, of course). Sorry, I read your words but have no clue as what you are trying to say with that statement. As i understand 'delta' to mean, the difference in file system content between a point in time 'A' and 'B' some point in time later in the future. Now just what is snapshot recording between point 'A' and 'B' and how does that apply to what dump is going to read and write? ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to freebsd-questions-unsubscr...@freebsd.org
Re: Dump questions
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 21/02/2010 12:52, Aiza wrote: Polytropon wrote: On Sun, 21 Feb 2010 11:42:50 +0800, Aiza aiz...@comclark.com wrote: 1. Using the -L flag to create a snapshot of the live running file system. Does this mean that a complete copy of the file system is written to .snap directory? No. The snapshot, quite incorrectly explained, is a saved delta between the file system on disk at a given state, to fixate further modifications (that are not included in the dump, of course). Sorry, I read your words but have no clue as what you are trying to say with that statement. As i understand 'delta' to mean, the difference in file system content between a point in time 'A' and 'B' some point in time later in the future. Now just what is snapshot recording between point 'A' and 'B' and how does that apply to what dump is going to read and write? In horrendously simplified terms, the way snapshots work is this. Whenever there would be a write to a disk block, instead of overwriting the original block, the content is copied and written out to a previously unused disk block. The original block is preserved temporarily while the snapshot is active -- so the snapshotted data you see is the comprised of: * All the disk blocks that haven't been altered during the lifetime of the snapshot * The original, unchanged disk blocks which have been replaced by modified copies in the live filesystem. ZFS always does the copy-on-write thing, so it's a very natural and very fast operation to create snapshots with it -- often described as 'snapshots for free' -- and you can have as many as you want. UFS doesn't do CoW by default (AFAIR) so creating a snapshot under UFS means toggling the default behaviour and initialising some data structures to keep track of the disk blocks that belong to each snapshot. This means it will take a few seconds to create and you can only have a limited number of snapshots per filesystem active simultaneously. In either case, the space used for the snapshot corresponds to the amount of changes made to the filesystem since the snapshot was created. Thus on an active fs, snapshot space usage will go up over time. However, the amount used will generally be a fairly small percentage of the total space on the device, and all the extra space is recovered when the snapshot is released. Cheers, Matthew - -- Dr Matthew J Seaman MA, D.Phil. 7 Priory Courtyard Flat 3 PGP: http://www.infracaninophile.co.uk/pgpkey Ramsgate Kent, CT11 9PW -BEGIN PGP SIGNATURE- Version: GnuPG/MacGPG2 v2.0.14 (Darwin) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/ iEYEARECAAYFAkuBNYQACgkQ8Mjk52CukIzETQCfdnu2W7BBRVrc1T2H3MPWMA1G KWsAnj6E2hZ3m2WTtMfTfqZ89sWzxaB8 =jOeb -END PGP SIGNATURE- ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to freebsd-questions-unsubscr...@freebsd.org
Re: Dump questions
On Sun, Feb 21, 2010 at 1:52 PM, Aiza aiz...@comclark.com wrote: Polytropon wrote: On Sun, 21 Feb 2010 11:42:50 +0800, Aiza aiz...@comclark.com wrote: 1. Using the -L flag to create a snapshot of the live running file system. Does this mean that a complete copy of the file system is written to .snap directory? No. The snapshot, quite incorrectly explained, is a saved delta between the file system on disk at a given state, to fixate further modifications (that are not included in the dump, of course). Sorry, I read your words but have no clue as what you are trying to say with that statement. As i understand 'delta' to mean, the difference in file system content between a point in time 'A' and 'B' some point in time later in the future. Now just what is snapshot recording between point 'A' and 'B' and how does that apply to what dump is going to read and write? A somewhat inaccurate explanation is this: When you (or dump -L) create a snapshot, the current state of the filesystem is frozen and a snapshot file is created. As soon as some process starts to modify the filesystem, the old blocks at the time of snapshot are written in the snapshot file and the new blocks are written in the (current) filesystem. Now, when you (or dump -L) read the snapshot file, the UFS filesystem does some magic behind the scenes: if the snapshot file contains the blocks you ask for, it means that the corresponding blocks in the filesystem have already been changed. That's okay: UFS will then give you the old blocks from the snapshot. If the snapshot file doesn't contain the blocks you asked for, it means that those blocks were not modified in the filesystem, so UFS gives you those unmodified blocks, right from the filesystem. When the snapshot file is deleted, the filesystem will not be monitored anymore and old blocks of modified blocks will not be saved anymore. The net result is that by reading the snapshot, you get the frozen state of the whole filesystem, while everybody else who does read the filesystem will get the current state. Well, it doesn't work exactly as outlined above, but conceptually, it comes pretty close. Regards, -cpghost. -- Cordula's Web. http://www.cordula.ws/ ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to freebsd-questions-unsubscr...@freebsd.org
Re: Dump questions
On Sun, Feb 21, 2010 at 02:10:29PM +0100, C. P. Ghost wrote: On Sun, Feb 21, 2010 at 1:52 PM, Aiza aiz...@comclark.com wrote: Polytropon wrote: On Sun, 21 Feb 2010 11:42:50 +0800, Aiza aiz...@comclark.com wrote: 1. Using the -L flag to create a snapshot of the live running file system. Does this mean that a complete copy of the file system is written to .snap directory? No. The snapshot, quite incorrectly explained, is a saved delta between the file system on disk at a given state, to fixate further modifications (that are not included in the dump, of course). Sorry, I read your words but have no clue as what you are trying to say with that statement. As i understand 'delta' to mean, the difference in file system content between a point in time 'A' and 'B' some point in time later in the future. Now just what is snapshot recording between point 'A' and 'B' and how does that apply to what dump is going to read and write? A somewhat inaccurate explanation is this: When you (or dump -L) create a snapshot, the current state of the filesystem is frozen and a snapshot file is created. As soon as some process starts to modify the filesystem, the old blocks at the time of snapshot are written in the snapshot file and the new blocks are written in the (current) filesystem. Now, when you (or dump -L) read the snapshot file, the UFS filesystem does some magic behind the scenes: if the snapshot file contains the blocks you ask for, it means that the corresponding blocks in the filesystem have already been changed. That's okay: UFS will then give you the old blocks from the snapshot. If the snapshot file doesn't contain the blocks you asked for, it means that those blocks were not modified in the filesystem, so UFS gives you those unmodified blocks, right from the filesystem. When the snapshot file is deleted, the filesystem will not be monitored anymore and old blocks of modified blocks will not be saved anymore. The net result is that by reading the snapshot, you get the frozen state of the whole filesystem, while everybody else who does read the filesystem will get the current state. Well, it doesn't work exactly as outlined above, but conceptually, it comes pretty close. Let me just add to this thread to make sure we document the intent: The point of such a snapshot is to give you a point-in-time consisent view of a filesystem. On an active system, the state of the file system may change from the time you beging the backup to the time you complete it, meaning that if you ever have to restore, you may find time skew corruption. By taking the snapshot, you at least have the point in time view of the file system. That doesn't mean it will be perfect! If there were files being updated or written at the moment the snapshot was taken, they may be internally inconsistent - part new, part old - but from a file system structure point of view, they will be uncorrupted. So - you may still have to take steps to queisce users, databases, or other applications to get everything in the file system in a known state BEFORE you take the snapshot! But, at least, you only need to do it for the few seconds it takes to instantiate the snapshot, and not the whole time it would take to do the backup. Without snapshot (to have a pefect backup): Shut down all your applications and kick off all your users If you're really paranoid, unmount the file system (so you know it's not being accessed) Run your dump remount the filesystem Restart your apps let your users back on You can have confidence when restoring from such a backup, but it may be a practical impossibility! With snapshot (and a few other gives to practicality) Alert your users of when the snapshot will begin, so they can avoid actively modifying files at the time of the snapshot Quiesce all your applications - many databases have ways that you can put them into extended logging mode or write-aside mode to give you a guaranteed, transactionally-consistent copy (and some just operate that way all the time), but if your applications don't USE transactions - well, it may be best to just shut them down for a few seconds anyway sync;sync;sync (OK - we probably don't do that anymore - I'm an old-timer) Start the snapshot Let everyone go back about their business Do your dump / backup Release the snapshot That should give you the same quality of backup, or pretty darn close, with minial interruption to your users and the services you provide. My point is, if you're simply starting the snapshot without doing anything else, you'll be in the same state after a restore as you would from a power failure or other system fault. If you are comfortable picking up from such a state and moving forward, that's fine, but if you need a more consistent state of affairs, you may want to take a few extra steps. Power outages and systems failures which stop filesystems cold are rare - but your snapshots and
Re: Dump questions
On Sun, 21 Feb 2010 20:52:31 +0800, Aiza aiz...@comclark.com wrote: Polytropon wrote: On Sun, 21 Feb 2010 11:42:50 +0800, Aiza aiz...@comclark.com wrote: 1. Using the -L flag to create a snapshot of the live running file system. Does this mean that a complete copy of the file system is written to .snap directory? No. The snapshot, quite incorrectly explained, is a saved delta between the file system on disk at a given state, to fixate further modifications (that are not included in the dump, of course). Sorry, I read your words but have no clue as what you are trying to say with that statement. As i understand 'delta' to mean, the difference in file system content between a point in time 'A' and 'B' some point in time later in the future. Now just what is snapshot recording between point 'A' and 'B' and how does that apply to what dump is going to read and write? Oh, I see I did express a bit unclear. The snapshot means that the filesystem's status of a certain point in time - here: when dump is beginning to run - is fixated in a snapshot file, representing its exact content at time A. This representation is subject to the dump. All further deltas after A are not incorporated into the snapshot, and of course not into the dump. This means that all changes after A are lost if the backup is restored. A welcome solution, especially when dumo + restore are used to transfer system and user data, is to first run dump and restore, and then use cpdup to commit changes that took place during or right after the dump to the target. -- Polytropon Magdeburg, Germany Happy FreeBSD user since 4.0 Andra moi ennepe, Mousa, ... ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to freebsd-questions-unsubscr...@freebsd.org
Re: Dump questions
On Sun, Feb 21, 2010 at 11:03:58AM +0100, Polytropon wrote: On Sun, 21 Feb 2010 11:42:50 +0800, Aiza aiz...@comclark.com wrote: 1. Using the -L flag to create a snapshot of the live running file system. ... Is this the limiting factor that forces a user to use (single user mode) for running dump? The snapshot, as far as the dump is concerned, essentially freezes the condition of the file system so that dump does not see any changes while dump is running. Without the snapshot, files could change or be deleted during the time it takes dump to finish. Dump starts by making its own list, by inode, of all the files to dump. Then it writes out, first the list, then the directories and finally the files and links to the media. If the files change between the time that list is made and things get written to the media, you will have an inaccurate representation of the file system. This can result in error messages if files it expects to be there are missing It can mean that a mangled image of a file is written in the dump. Doing a dump in Single User Mode means stopping activity on the system so there are fewer chances of the above happening. Using -L and doing a snapshot will not prevent a dump from being technically obsolete by the time it gets done, but it will mean that what gets written to media is internally consistent. The list it made will be exactly what is on the backup media and the files are all written completely as they were when the snapshot was taken with no mangling. 2. What is the worse that will happen if dump is run on live file system with out the -L flag? The index list that is written as part of the dump will not reflect what is on the dump media. It may claim a file is there, but it really is not. A file or some files are mangled because they are open and being modified by another process as the dump is reading them. The file may be either an inaccurate image or even completely unreadable. Restore is smart enough to skip over these problems if the file[s] you are looking to restore are not the ones mangled or deleted. But, you could get in to a situation of not being able to restore some things that you have on media. Can dump recognize this situation and issue an error message? I don't remember if dump puts out any useful diagnostic. I think it might tell you if it cannot file a file whose inode is in its list to write. But, it is restore that really notices and complains. If you have room, you can use restore to 'verify' a dump just by doing a restore of it to some extra space (maybe even to /dev/null, though I have never tried that one) and seeing if it makes any complaints. This, of course, is a long way to do this, but it might be valuable if it is essential for that dump to be completely readable in a later situation where the original is not longer available. But, in this situation, then making a -L dump (using a snapshot) is really important or even a single user, filesystem unmounted -L dump. 3. Can dump be told to only dump a particular directory tree? IE /var/log or /usr/port? dump only workes on filesystems/partitions. If you know you will want to make dumps on just that directory tree, that is a reason to make a separate partition/filesystem for it and mount it up. There is no reason that /var/log cannot be in its own partition/filesystem separate from /var and just mounted that way. Of course, you have to make sure that /var gets mounted before /var/log. But, that is not strange. Many people make a separate partition for /usr/home inside of /usr or a /var/db that is mounted inside of /var. Now, you can restore just a single file, group of files or a directory tree out of a dump. You do not have to restore the whole dump. So, you can make a dump of a '/var' filesystem if that is what you have and then if you need to restore just '/var/db' out of it, that is no problem. Just make sure you understand where you are putting it and how you specifiy it in the restore. But, if you just want a backup copy of a directory tree that is not its own partition/filesystem, you must use some other tool, such as tar or possibly rsync. jerry ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to freebsd-questions-unsubscr...@freebsd.org ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to freebsd-questions-unsubscr...@freebsd.org
Re: Dump questions
Jerry McAllister wrote: On Sun, Feb 21, 2010 at 11:03:58AM +0100, Polytropon wrote: On Sun, 21 Feb 2010 11:42:50 +0800, Aiza aiz...@comclark.com wrote: 1. Using the -L flag to create a snapshot of the live running file system. ... Is this the limiting factor that forces a user to use (single user mode) for running dump? The snapshot, as far as the dump is concerned, essentially freezes the condition of the file system so that dump does not see any changes while dump is running. Without the snapshot, files could change or be deleted during the time it takes dump to finish. Dump starts by making its own list, by inode, of all the files to dump. Then it writes out, first the list, then the directories and finally the files and links to the media. If the files change between the time that list is made and things get written to the media, you will have an inaccurate representation of the file system. This can result in error messages if files it expects to be there are missing It can mean that a mangled image of a file is written in the dump. Doing a dump in Single User Mode means stopping activity on the system so there are fewer chances of the above happening. Using -L and doing a snapshot will not prevent a dump from being technically obsolete by the time it gets done, but it will mean that what gets written to media is internally consistent. The list it made will be exactly what is on the backup media and the files are all written completely as they were when the snapshot was taken with no mangling. 2. What is the worse that will happen if dump is run on live file system with out the -L flag? The index list that is written as part of the dump will not reflect what is on the dump media. It may claim a file is there, but it really is not. A file or some files are mangled because they are open and being modified by another process as the dump is reading them. The file may be either an inaccurate image or even completely unreadable. Restore is smart enough to skip over these problems if the file[s] you are looking to restore are not the ones mangled or deleted. But, you could get in to a situation of not being able to restore some things that you have on media. Can dump recognize this situation and issue an error message? I don't remember if dump puts out any useful diagnostic. I think it might tell you if it cannot file a file whose inode is in its list to write. But, it is restore that really notices and complains. If you have room, you can use restore to 'verify' a dump just by doing a restore of it to some extra space (maybe even to /dev/null, though I have never tried that one) and seeing if it makes any complaints. This, of course, is a long way to do this, but it might be valuable if it is essential for that dump to be completely readable in a later situation where the original is not longer available. But, in this situation, then making a -L dump (using a snapshot) is really important or even a single user, filesystem unmounted -L dump. 3. Can dump be told to only dump a particular directory tree? IE /var/log or /usr/port? dump only workes on filesystems/partitions. If you know you will want to make dumps on just that directory tree, that is a reason to make a separate partition/filesystem for it and mount it up. There is no reason that /var/log cannot be in its own partition/filesystem separate from /var and just mounted that way. Of course, you have to make sure that /var gets mounted before /var/log. But, that is not strange. Many people make a separate partition for /usr/home inside of /usr or a /var/db that is mounted inside of /var. Now, you can restore just a single file, group of files or a directory tree out of a dump. You do not have to restore the whole dump. So, you can make a dump of a '/var' filesystem if that is what you have and then if you need to restore just '/var/db' out of it, that is no problem. Just make sure you understand where you are putting it and how you specifiy it in the restore. But, if you just want a backup copy of a directory tree that is not its own partition/filesystem, you must use some other tool, such as tar or possibly rsync. jerry Thank you for the detail insight of how dump functions. Now one more question. Is the dump -L backup file made in a multiple-user-mode environment just as dependable as dump backups made in single-user-mode? ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to freebsd-questions-unsubscr...@freebsd.org
Dump questions
1. Using the -L flag to create a snapshot of the live running file system. Does this mean that a complete copy of the file system is written to .snap directory? So if the running file system is more than 50% full there will not be enough free space available to hold the duplicate image? Can dump recognize this situation and issue an error message? Is this the limiting factor that forces a user to use (single user mode) for running dump? 2. What is the worse that will happen if dump is run on live file system with out the -L flag? Can dump recognize this situation and issue an error message? 3. Can dump be told to only dump a particular directory tree? IE /var/log or /usr/port? ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to freebsd-questions-unsubscr...@freebsd.org
Re: Dump questions
In the last episode (Feb 21), Aiza said: 1. Using the -L flag to create a snapshot of the live running file system. Does this mean that a complete copy of the file system is written to .snap directory? No; that would be a copy. Snapshots only copy blocks as they are modified on the parent filesystem, so their size is determined by how much data is modified since the snapshot was created. -- Dan Nelson dnel...@allantgroup.com ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to freebsd-questions-unsubscr...@freebsd.org
Re: Dump questions
3. Can dump be told to only dump a particular directory tree? IE /var/log or /usr/port? No. ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to freebsd-questions-unsubscr...@freebsd.org
Re: Dump questions
Dan Nelson wrote: In the last episode (Feb 21), Aiza said: 1. Using the -L flag to create a snapshot of the live running file system. Does this mean that a complete copy of the file system is written to .snap directory? No; that would be a copy. Snapshots only copy blocks as they are modified on the parent filesystem, so their size is determined by how much data is modified since the snapshot was created. So how does this interact with the dump process? Dump start reading and writing its dump file and as the live system changes the changes are written to the .snap and when dump completes it overwrites it dump with the changes from the .snap??? How does this process work in detail? ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to freebsd-questions-unsubscr...@freebsd.org
Re: Dump questions
In the last episode (Feb 21), Aiza said: Dan Nelson wrote: In the last episode (Feb 21), Aiza said: 1. Using the -L flag to create a snapshot of the live running file system. Does this mean that a complete copy of the file system is written to .snap directory? No; that would be a copy. Snapshots only copy blocks as they are modified on the parent filesystem, so their size is determined by how much data is modified since the snapshot was created. So how does this interact with the dump process? Dump start reading and writing its dump file and as the live system changes the changes are written to the .snap and when dump completes it overwrites it dump with the changes from the .snap??? How does this process work in detail? Dump reads from the snapshot, which is guaranteed not to change while dump is running. When its done, dump deletes the snapshot file. Changes made after the dump has started will not be saved. This is the same as any other backup system that uses snapshots afaik; none try and catch up changes made while the backup itself is running. You could run another incremental dump right after the previous one, which would back up any changes since the first one. -- Dan Nelson dnel...@allantgroup.com ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to freebsd-questions-unsubscr...@freebsd.org
