Re: Hot-backup a complete Debian install
On Wed, Oct 01, 2003 at 11:52:29PM +0900, Nathan Ollerenshaw wrote: > On Oct 1, 2003, at 10:51 PM, DI Peter Burgstaller wrote: > > >That is exactly the beauty of dump. I would have suggested dd for > >backup/restore but there > >you have the problem of identical fdisk settings. Dump/restore can > >deal well with bigger partitions. > > Definitely use dump. > > Its much faster than anything else and if you really learn how to use > it, you can do incremental backups very quickly. Just quickly, does dump/restore's incremental backups do correct system "snapshots", accurately recording deleted/renamed etc files, or does it just backup files with ctimes/mtimes greater than the last backup? Last time I looked, which was admittedly a long time ago, very few tools did incremental backups "properly". You couldn't accurately restore a full system from an incremental backup... you would get extra files that had been deleted by the time of the incremental backup. Even worse, many backup tools just use mtime/ctime to detect "changed files", which misses renamed files. I'm pretty sure dump/restore was one of the ones that didn't do it properly. At the time one of the few that did incrementals right was a little shell script called "tob". I believe another more recent tool that does is rdiff-backup. -- Donovan Baardahttp://minkirri.apana.org.au/~abo/ -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of "unsubscribe". Trouble? Contact [EMAIL PROTECTED]
Re: Hot-backup a complete Debian install
On Wed, 1 Oct 2003 10:05:50 +0200, you wrote: >we're using a similar setup for some hosts and I have the best results >so far with dump/restore on ext2/ext3 partitions. Also using snapshots with dump/restore? Do you have any script which I could have a look? >I've even successfully recreated a database server with mysql and >postgresql servers running >and using dump as a backup tool. My idea is generate a different backup for databases, using especific MySQL&co tools. I don't think a backup of a live filesystem containing database data could be reliable.. what happens if MySQL has some data in memory (still not flushed to disk)? Saludos, --Roman -- PGP Fingerprint: 09BB EFCD 21ED 4E79 25FB 29E1 E47F 8A7D EAD5 6742 [Key ID: 0xEAD56742. Available at KeyServ] -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of "unsubscribe". Trouble? Contact [EMAIL PROTECTED]
Re: Hot-backup a complete Debian install
Looking through the debian package list I have noticed this a while ago: afbackup This is a client-server backup system offering several workstations a centralized backup to a special backup server. Backing up only one computer is easily possible, too. Any streaming device can be used for writing the data to it, usually this will be a tape device. Writing backups is normally done sequentially: The next writing to tape goes to the end of the previous write no matter where you have restored from in the meantime. Has anyone ever tried it? Regarding your question I would vote for tar too - though rsync and dump (never used - doesn't work with reiserfs) seem to be used a lot of other people. Actually, looking through the debian package list produces quite a number of interesting things.. Michael Moritz Roman Medina wrote: Hi, I'd like to know which tools&methods do you prefer for backing up a complete Linux install _in a production environment_, i.e., _without having to shut down the machine or unmount partitions_. The machine needs to be always alive and it will be remotely administered. I'd like to hear your opinion about the following idea. 1) I have LVM + ext3, with a distribution like: linux:~# mount /dev/hda3 on / type ext3 (rw,errors=remount-ro) proc on /proc type proc (rw) devpts on /dev/pts type devpts (rw,gid=5,mode=620) /dev/rs/usr on /usr type ext3 (rw,nodev) /dev/rs/var on /var type ext3 (rw,nodev,usrquota) /dev/rs/tmp on /tmp type ext3 (rw,nosuid,nodev,usrquota) /dev/rs/www on /www type ext3 (rw,nosuid,nodev,usrquota) /dev/rs/ftp on /ftp type ext3 (rw,nosuid,nodev,usrquota) /dev/rs/mail on /mail type ext3 (rw,nosuid,nodev,usrquota) /dev/hda2 on /boot type ext3 (rw) rs-hosting:~# 2) Root and boot partitions are not LVM based, but they're quite static, since data and logs are stored in other partitions. Is it safe to use "dump" here (keeping the partition mounted)? 3) All the "dynamic" partitions are LVM-based, so the idea is taking one snapshot of each partition and use "dump" util over the snapshot? Is it a good idea? Would it work? Please, I'd like to receive comments / another ideas on this issue. Thanks. Saludos, --Roman -- PGP Fingerprint: 09BB EFCD 21ED 4E79 25FB 29E1 E47F 8A7D EAD5 6742 [Key ID: 0xEAD56742. Available at KeyServ] -- Please note that this account is being filtered using anti UCE systems. If you send email to this account make sure that it could not be mistaken as UCE. -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of "unsubscribe". Trouble? Contact [EMAIL PROTECTED]
Re: Hot-backup a complete Debian install
>> Finally, the 3rd stage: if you're going to save the backup files in an >> "non-trusty" machine, which kind of container / encryption software >> would you use? This would need to be easily scriptable, for automatize >> the backup task. My idea is to use something like BestCrypt: http://www.jetico.com/linux/ Then you can mount the virtual container, write the dump file/s into it, unmount and send the virtual container thrught the network. You could use (non-encrypted) ftp, since the container by itself is already encrypted. > Hmm .. tricky .. anything that is run by a script .. has the problem > that if the > script can be read .. anybody can do what the script does. No, the point here is that only the destination machine (which is going to be used as "backup store") is untrusty. Anyway, if a person have root on the origin machine, he/she doesn't need to "hack" the container files nor the backup files: he/she can get the files directly from the live system. > Another option to consider is that those dump files will be big > (depending on your system) > Running them through a gpg -e will a) make the machine slow and b) take > a LONG time. Yes, but my former scheme (BestCrypt or similar): - you save the CPU used to SSL/SSH the transfers over the net (since you could simply ftp the container).- when you have the container mounted and you're dumping into it the encryption is made in a realtime way, so perhaps the bottleneck is not CPU but hard-disk usage (since "dump" util is reading the filesystem in parallel).- you could use "nice" so the process only uses free CPY cycles. > I'd be very interested on how to solve this one .. I have another problem with getting LVM snapshots to work: linux:~/backup# lvcreate -L592M -s -n backup-ftp /dev/rs/ftp lvcreate -- WARNING: the snapshot will be automatically disabled once it gets fulllvcreate -- INFO: using default snapshot chunk size of 64 KB for "/dev/rs/backup-ftp"lvcreate -- doing automatic backup of "rs" lvcreate -- logical volume "/dev/rs/backup-ftp" successfully created linux:~/backup# mount /dev/rs/backup-ftp /mnt mount: block device /dev/rs/backup-ftp is write-protected, mounting read-only mount: wrong fs type, bad option, bad superblock on /dev/rs/backup-ftp, or too many mounted file systems linux:~/backup# uname -a Linux linux 2.4.20-bf2.4 #1 Wed Dec 25 13:17:08 UTC 2002 i686 unknown linux:~/backup# I googled a bit and I've read some comments about similar problem. I think that I'll have to recompile my kernel but I don't know which exact option would I have to enable, or if this would be the right solution. Any details which could help me? -R -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of "unsubscribe". Trouble? Contact [EMAIL PROTECTED]
Re: Hot-backup a complete Debian install
On Oct 1, 2003, at 10:51 PM, DI Peter Burgstaller wrote: That is exactly the beauty of dump. I would have suggested dd for backup/restore but there you have the problem of identical fdisk settings. Dump/restore can deal well with bigger partitions. Definitely use dump. Its much faster than anything else and if you really learn how to use it, you can do incremental backups very quickly. And, like Peter is saying, it will work better when you have to restore to another machine with different disk sizes. BTW, if you want a commercial solution, look at CA's ARCserve or Legato's Networker, both are good products that support Linux. Can be a little expensive if you're on a budget though. Regards, Nathan. -- Nathan Ollerenshaw - Unix Systems Engineer ValueCommerce - http://www.valuecommerce.ne.jp/ -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of "unsubscribe". Trouble? Contact [EMAIL PROTECTED]
Re: Hot-backup a complete Debian install
finally, if you dont trust the machine you must encrypt before transfer :-| lot of time, cpu and disk. another option could be sign the backup with md5 for example? this way you can trust the backup content. i think... Roman Medina <[EMAIL PROTECTED]> dijo: > On Wed, 1 Oct 2003 11:42:25 -, you wrote: > > >which is the backup target media? > > Hard-disk. The idea is to have another logical partition for backups > and then some scripts to upload/download to any secure site (I could > use rsync over ssh or simply scp). But the uploading is a second step. > Now I'd like to deal with the first stage: the dump process. > > Another question: for restoring a backup, do I need to create (fdisk) > a destination partition with the exact same size of the original? > Could I use a greater one? (in this case, would I lose disk-space or > the filesystem is "expanded" accordingly?). > > Finally, the 3rd stage: if you're going to save the backup files in > an "non-trusty" machine, which kind of container / encryption software > would you use? This would need to be easily scriptable, for automatize > the backup task. > > Thanks to all the contributors of this thread! > > Regards, > --Roman > > -- > PGP Fingerprint: > 09BB EFCD 21ED 4E79 25FB 29E1 E47F 8A7D EAD5 6742 > [Key ID: 0xEAD56742. Available at KeyServ] > > > -- > To UNSUBSCRIBE, email to [EMAIL PROTECTED] > with a subject of "unsubscribe". Trouble? Contact [EMAIL PROTECTED] > -- -- Creado con Evo-GroupWare Evolution WebServices - www.evolution.com.uy -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of "unsubscribe". Trouble? Contact [EMAIL PROTECTED]
Re: Hot-backup a complete Debian install
first of all, sorry G i can't understand your point. well, i can tell you about one of my experiences that can explain some points of your question. i had a machine with a scsi disk and linux installed, someday i started to experience some i/o errors on that disk so i have to migrate the instalation. so, i was looking for another scsi disk, but i only found an ide one. the result was: _ i backup the scsi partitions in the ide partitions (made by hand, equal or greater size) with tar. _ modified the fstab to match the root and the others new devices with mount points.(/dev/sdax --> /dev/hdax) _ update the bootloader and mbr!! _ reboot.. .. and voila! all in the last place before the errors... it's not lvm but i hope some concepts helps you. if the point is about file level backup tar (i think) is good. a block level backup could be 'dd' good luck. Roman Medina <[EMAIL PROTECTED]> dijo: > On Wed, 1 Oct 2003 11:42:25 -, you wrote: > > >which is the backup target media? > > Hard-disk. The idea is to have another logical partition for backups > and then some scripts to upload/download to any secure site (I could > use rsync over ssh or simply scp). But the uploading is a second step. > Now I'd like to deal with the first stage: the dump process. > > Another question: for restoring a backup, do I need to create (fdisk) > a destination partition with the exact same size of the original? > Could I use a greater one? (in this case, would I lose disk-space or > the filesystem is "expanded" accordingly?). > > Finally, the 3rd stage: if you're going to save the backup files in > an "non-trusty" machine, which kind of container / encryption software > would you use? This would need to be easily scriptable, for automatize > the backup task. > > Thanks to all the contributors of this thread! > > Regards, > --Roman > > -- > PGP Fingerprint: > 09BB EFCD 21ED 4E79 25FB 29E1 E47F 8A7D EAD5 6742 > [Key ID: 0xEAD56742. Available at KeyServ] > > > -- > To UNSUBSCRIBE, email to [EMAIL PROTECTED] > with a subject of "unsubscribe". Trouble? Contact [EMAIL PROTECTED] > -- -- Creado con Evo-GroupWare Evolution WebServices - www.evolution.com.uy -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of "unsubscribe". Trouble? Contact [EMAIL PROTECTED]
Re: Hot-backup a complete Debian install
On Wednesday, October 1, 2003, at 3:43 PM, Roman Medina wrote: On Wed, 1 Oct 2003 11:42:25 -, you wrote: which is the backup target media? Hard-disk. The idea is to have another logical partition for backups and then some scripts to upload/download to any secure site (I could use rsync over ssh or simply scp). But the uploading is a second step. Now I'd like to deal with the first stage: the dump process. Another question: for restoring a backup, do I need to create (fdisk) a destination partition with the exact same size of the original? Could I use a greater one? (in this case, would I lose disk-space or the filesystem is "expanded" accordingly?). That is exactly the beauty of dump. I would have suggested dd for backup/restore but there you have the problem of identical fdisk settings. Dump/restore can deal well with bigger partitions. Finally, the 3rd stage: if you're going to save the backup files in an "non-trusty" machine, which kind of container / encryption software would you use? This would need to be easily scriptable, for automatize the backup task. Hmm .. tricky .. anything that is run by a script .. has the problem that if the script can be read .. anybody can do what the script does. Another option to consider is that those dump files will be big (depending on your system) Running them through a gpg -e will a) make the machine slow and b) take a LONG time. I'd be very interested on how to solve this one .. - Cheers, Peter -- Dipl.-Ing. Peter Burgstaller Technical Director @ all information network & services gmbh email: [EMAIL PROTECTED] phone: +43 662 452335 fax : +43 662 452335 90 -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of "unsubscribe". Trouble? Contact [EMAIL PROTECTED]
Re: Hot-backup a complete Debian install
On Wed, 1 Oct 2003 11:42:25 -, you wrote: >which is the backup target media? Hard-disk. The idea is to have another logical partition for backups and then some scripts to upload/download to any secure site (I could use rsync over ssh or simply scp). But the uploading is a second step. Now I'd like to deal with the first stage: the dump process. Another question: for restoring a backup, do I need to create (fdisk) a destination partition with the exact same size of the original? Could I use a greater one? (in this case, would I lose disk-space or the filesystem is "expanded" accordingly?). Finally, the 3rd stage: if you're going to save the backup files in an "non-trusty" machine, which kind of container / encryption software would you use? This would need to be easily scriptable, for automatize the backup task. Thanks to all the contributors of this thread! Regards, --Roman -- PGP Fingerprint: 09BB EFCD 21ED 4E79 25FB 29E1 E47F 8A7D EAD5 6742 [Key ID: 0xEAD56742. Available at KeyServ] -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of "unsubscribe". Trouble? Contact [EMAIL PROTECTED]
Re: Hot-backup a complete Debian install
Alejandro Vartabedian wrote: the hole file system! That must be /dev/null ! G -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of "unsubscribe". Trouble? Contact [EMAIL PROTECTED]
Re: Hot-backup a complete Debian install
what about tar? it's a good tool, support device files, opened files, etc. the hole file system! which is the backup target media? good luck. pd: take a look in `man tar` Roman Medina <[EMAIL PROTECTED]> dijo: > > Hi, > > I'd like to know which tools&methods do you prefer for backing up a > complete Linux install _in a production environment_, i.e., _without > having to shut down the machine or unmount partitions_. The machine > needs to be always alive and it will be remotely administered. > > I'd like to hear your opinion about the following idea. > 1) I have LVM + ext3, with a distribution like: > linux:~# mount > /dev/hda3 on / type ext3 (rw,errors=remount-ro) > proc on /proc type proc (rw) > devpts on /dev/pts type devpts (rw,gid=5,mode=620) > /dev/rs/usr on /usr type ext3 (rw,nodev) > /dev/rs/var on /var type ext3 (rw,nodev,usrquota) > /dev/rs/tmp on /tmp type ext3 (rw,nosuid,nodev,usrquota) > /dev/rs/www on /www type ext3 (rw,nosuid,nodev,usrquota) > /dev/rs/ftp on /ftp type ext3 (rw,nosuid,nodev,usrquota) > /dev/rs/mail on /mail type ext3 (rw,nosuid,nodev,usrquota) > /dev/hda2 on /boot type ext3 (rw) > rs-hosting:~# > > 2) Root and boot partitions are not LVM based, but they're quite > static, since data and logs are stored in other partitions. Is it safe > to use "dump" here (keeping the partition mounted)? > > 3) All the "dynamic" partitions are LVM-based, so the idea is taking > one snapshot of each partition and use "dump" util over the snapshot? > Is it a good idea? Would it work? > > Please, I'd like to receive comments / another ideas on this issue. > Thanks. > > Saludos, > --Roman > > -- > PGP Fingerprint: > 09BB EFCD 21ED 4E79 25FB 29E1 E47F 8A7D EAD5 6742 > [Key ID: 0xEAD56742. Available at KeyServ] > > > -- > To UNSUBSCRIBE, email to [EMAIL PROTECTED] > with a subject of "unsubscribe". Trouble? Contact [EMAIL PROTECTED] > -- -- Creado con Evo-GroupWare Evolution WebServices - www.evolution.com.uy -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of "unsubscribe". Trouble? Contact [EMAIL PROTECTED]
Re: Hot-backup a complete Debian install
> I might be (and probably am) missing something, but wouldn't rsync (over > ssh) work? Does rsync handle device files correctly, hard links, sparse files, etc? (I'm not trying to troll, I honestly don't know if it handles all of these things.) I saw an article a while back comparing backup tools, I don't recall where, that basically concluded that the only tool that was safe for full backup/restore was "dump" and "restore". Even tar didn't handle every file type correctly (I think it didn't do sparse files - it ended up inflating them) - I don't recall if they tested "star" or not. Anyways, since then I've been meaning to look into dump/restore but have not yet. Take care, Dale -- Dale E. Martin, Clifton Labs, Inc. Senior Computer Engineer [EMAIL PROTECTED] http://www.cliftonlabs.com pgp key available -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of "unsubscribe". Trouble? Contact [EMAIL PROTECTED]
Re: Hot-backup a complete Debian install
Hi, we're using a similar setup for some hosts and I have the best results so far with dump/restore on ext2/ext3 partitions. I've even successfully recreated a database server with mysql and postgresql servers running and using dump as a backup tool. No problems so far. - Cheers, Peter -- Dipl.-Ing. Peter Burgstaller Technical Director @ all information network & services gmbh email: [EMAIL PROTECTED] phone: +43 662 452335 fax : +43 662 452335 90 -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of "unsubscribe". Trouble? Contact [EMAIL PROTECTED]
Re: Hot-backup a complete Debian install
Hi, I might be (and probably am) missing something, but wouldn't rsync (over ssh) work? Steve On Wed, Oct 01, 2003 at 01:00:20AM +0200, Roman Medina wrote: > > Hi, > > I'd like to know which tools&methods do you prefer for backing up a > complete Linux install _in a production environment_, i.e., _without > having to shut down the machine or unmount partitions_. The machine > needs to be always alive and it will be remotely administered. > > I'd like to hear your opinion about the following idea. > 1) I have LVM + ext3, with a distribution like: > linux:~# mount > /dev/hda3 on / type ext3 (rw,errors=remount-ro) > proc on /proc type proc (rw) > devpts on /dev/pts type devpts (rw,gid=5,mode=620) > /dev/rs/usr on /usr type ext3 (rw,nodev) > /dev/rs/var on /var type ext3 (rw,nodev,usrquota) > /dev/rs/tmp on /tmp type ext3 (rw,nosuid,nodev,usrquota) > /dev/rs/www on /www type ext3 (rw,nosuid,nodev,usrquota) > /dev/rs/ftp on /ftp type ext3 (rw,nosuid,nodev,usrquota) > /dev/rs/mail on /mail type ext3 (rw,nosuid,nodev,usrquota) > /dev/hda2 on /boot type ext3 (rw) > rs-hosting:~# > > 2) Root and boot partitions are not LVM based, but they're quite > static, since data and logs are stored in other partitions. Is it safe > to use "dump" here (keeping the partition mounted)? > > 3) All the "dynamic" partitions are LVM-based, so the idea is taking > one snapshot of each partition and use "dump" util over the snapshot? > Is it a good idea? Would it work? > > Please, I'd like to receive comments / another ideas on this issue. > Thanks. > > Saludos, > --Roman > > -- > PGP Fingerprint: > 09BB EFCD 21ED 4E79 25FB 29E1 E47F 8A7D EAD5 6742 > [Key ID: 0xEAD56742. Available at KeyServ] > > > -- > To UNSUBSCRIBE, email to [EMAIL PROTECTED] > with a subject of "unsubscribe". Trouble? Contact [EMAIL PROTECTED] -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of "unsubscribe". Trouble? Contact [EMAIL PROTECTED]
