Re: [BackupPC-users] Restoring complete virtualized Windows-Servers / Saving MBR
I'm working with remote backups of large files, and found a solution that can be used when the backup file is quite similar to the previous one. I think that it could be used to send by rsync large backups of virtual disks. Basically, you have to: * gzip --rsyncable today_backup_file (very important to use --rsyncable) * rollcut -d 65536 today_backup_file.gzip * dupemerge backup_folder (deduplication using hardlinks) * use backuppc and rsync to remote backup your backup_folder Rollcut is a tool that I've done to cut similar large files, into several small chunks with a hig probability of some being equal. After using rollcut, you deduplicate all the small chunks, using hardlinks. So, every chunk that is the same of one of the previous ones, will be hardlinked locally. As rsync knows about hardlinks, after the first backup all the hardlinked chunks will not have to be send over the network. You can find more information, and download rollcut in https://github.com/ferfebles/rollcut -- Live Security Virtual Conference Exclusive live event will cover all the ways today's security and threat landscape has changed and how IT managers can respond. Discussions will include endpoint security, mobile security and the latest in malware threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/ ___ BackupPC-users mailing list BackupPC-users@lists.sourceforge.net List:https://lists.sourceforge.net/lists/listinfo/backuppc-users Wiki:http://backuppc.wiki.sourceforge.net Project: http://backuppc.sourceforge.net/
Re: [BackupPC-users] Restoring complete virtualized Windows-Servers / Saving MBR
Hi Gerald, urbackup is completely new to me. The features sounds exactly like what I need. The website and documentation doesn't look that evolved from the first sight, at least compared to backupPC. But I think I just need to try the software out: The setup looks easy and I think I can do a backup/restore test with a naked windows xp in a VirtualBox. Thank you very much for the hint, Andreas Am 16.04.2012 um 14:16 schrieb Gerald Brandt: > Hi, > > I use BackupPC for backups, and Urbackup for disaster recovery, imaging the > boot drive only. BackupPC does the data drive. > > http://www.urbackup.org/ > > Gerald > > > Two completely separate backup schemes are needed here. > > One for full "cold-metal" restores of the boot/OS level stuff, and IMO this > is best done with "imaging" style software, in your case specifically > targeted for windoze/ntfs systems. These don't need to be done very > frequently, as little is changing from day to day. BPC is not intended to > provide for this kind of backup, especially regarding Windows. Many Linux > sysadmins simply re-install their OS from automated scripts and then restore > config files rather than bothering to fully image their boot/OS partitions, > but Windows isn't suited to that approach. > > The type of backup is for working data, which requires the frequent > full/incremental/archive that BPC is designed for. Details about the > infrastructure "under" the filesystem are irrelevant to BPC, except when > considering how to optimize performance when a small "backup window" becomes > an issue. > > What you are doing with LVM snapshotting should only be necessary for certain > applications that keep their files open, like DB servers, Outlook and some > FOSS mailsystems. And then only if these services need to be kept running as > close to 24/7 as possible, otherwise your scripts can just close the server > processes down until the backup is complete and then bring them back up again. > > I can't advise on the NTFS-specific extended attributes and newfangled > security stuff, but unless you're using software that specifically leverages > that MS-proprietary stuff, it shouldn't IMO be an issue. > > > -- > For Developers, A Lot Can Happen In A Second. > Boundary is the first to Know...and Tell You. > Monitor Your Applications in Ultra-Fine Resolution. Try it FREE! > http://p.sf.net/sfu/Boundary-d2dvs2 > > ___ > BackupPC-users mailing list > BackupPC-users@lists.sourceforge.net > List:https://lists.sourceforge.net/lists/listinfo/backuppc-users > Wiki:http://backuppc.wiki.sourceforge.net > Project: http://backuppc.sourceforge.net/ > > -- > For Developers, A Lot Can Happen In A Second. > Boundary is the first to Know...and Tell You. > Monitor Your Applications in Ultra-Fine Resolution. Try it FREE! > http://p.sf.net/sfu/Boundary-d2dvs2 > ___ > BackupPC-users mailing list > BackupPC-users@lists.sourceforge.net > List:https://lists.sourceforge.net/lists/listinfo/backuppc-users > Wiki:http://backuppc.wiki.sourceforge.net > Project: http://backuppc.sourceforge.net/ -- For Developers, A Lot Can Happen In A Second. Boundary is the first to Know...and Tell You. Monitor Your Applications in Ultra-Fine Resolution. Try it FREE! http://p.sf.net/sfu/Boundary-d2dvs2 ___ BackupPC-users mailing list BackupPC-users@lists.sourceforge.net List:https://lists.sourceforge.net/lists/listinfo/backuppc-users Wiki:http://backuppc.wiki.sourceforge.net Project: http://backuppc.sourceforge.net/
Re: [BackupPC-users] Restoring complete virtualized Windows-Servers / Saving MBR
- Original Message - > From: "Andreas Piening" > One thing you stated about doing virtual disk image backups is that > they're huge. As soon as anything in the virtual machine changes, > the complete image needs to be stored again to keep the image backup > in sync. That's what I want to avoid by using LVM volumes: I can > create a R/O snapshot of them and mount them into the filesystem > just like an external harddrive connected to the system. From there > I can do a file-based backups which is way more space efficient. I tried doing something like this once upon a time. Taking an LVM snapshot on an active disk system will kill your performance. It may work, but you may find that it's terribly slow. It's best to back up virtual machines by connecting to the virtual machine itself and backing up from there. -- Carl Soderstrom Systems Administrator Real-Time Enterprises www.real-time.com -- For Developers, A Lot Can Happen In A Second. Boundary is the first to Know...and Tell You. Monitor Your Applications in Ultra-Fine Resolution. Try it FREE! http://p.sf.net/sfu/Boundary-d2dvs2 ___ BackupPC-users mailing list BackupPC-users@lists.sourceforge.net List:https://lists.sourceforge.net/lists/listinfo/backuppc-users Wiki:http://backuppc.wiki.sourceforge.net Project: http://backuppc.sourceforge.net/
Re: [BackupPC-users] Restoring complete virtualized Windows-Servers / Saving MBR
On 16/04/12 11:50, Andreas Piening wrote: > Hi, > > thank you for your response. > > At the moment I don't use disk images. Instead I use LVM volumes which are > directly connected to my KVM-machines. > There is a way like creating images of the LVM volumes with a image tool like > partimage. These images would be compressed like 50GB in combined size, no > problem to copy it to an external usb drive but much data to transfer over a > VDSL50 internet connection. > The point is that the customer wants me to backup the whole system > (real-server including 2 VMs) over the network to a different location. So > when the server gets unusable damaged for instance by beeing flooded with > water because of a pipe-break, or the system gets stolen, I should be able to > buy new hardware and get back to the state one day before the disaster > occurred. > > I like the efficient way of file based backups backupPC uses, so I ask for > experiences on that. But maybe I should search for a partition image tool > that supports incremental backups. I only know of Acronis True Image but this > is a commercial (and not cheap) way. If you rip the LVM disk images to a file backuppc will just backup the changes as that's how rsync works, however this requires you to the storage to image the disk to a working location. Your other option if you are looking for disaster recovery is something like DRDB and a live snapshot of the VMs to a remote location. This is not backup as the remote image is updated live so make a change and it's replicated to the remote location very quickly, but it is a DR option. -- Tim Fletcher -- For Developers, A Lot Can Happen In A Second. Boundary is the first to Know...and Tell You. Monitor Your Applications in Ultra-Fine Resolution. Try it FREE! http://p.sf.net/sfu/Boundary-d2dvs2 ___ BackupPC-users mailing list BackupPC-users@lists.sourceforge.net List:https://lists.sourceforge.net/lists/listinfo/backuppc-users Wiki:http://backuppc.wiki.sourceforge.net Project: http://backuppc.sourceforge.net/
Re: [BackupPC-users] Restoring complete virtualized Windows-Servers / Saving MBR
Hi, I use BackupPC for backups, and Urbackup for disaster recovery, imaging the boot drive only. BackupPC does the data drive. http://www.urbackup.org/ Gerald - Original Message - > Two completely separate backup schemes are needed here. > One for full "cold-metal" restores of the boot/OS level stuff, and > IMO this is best done with "imaging" style software, in your case > specifically targeted for windoze/ntfs systems. These don't need to > be done very frequently, as little is changing from day to day. BPC > is not intended to provide for this kind of backup, especially > regarding Windows. Many Linux sysadmins simply re-install their OS > from automated scripts and then restore config files rather than > bothering to fully image their boot/OS partitions, but Windows isn't > suited to that approach. > The type of backup is for working data, which requires the frequent > full/incremental/archive that BPC is designed for. Details about the > infrastructure "under" the filesystem are irrelevant to BPC, except > when considering how to optimize performance when a small "backup > window" becomes an issue. > What you are doing with LVM snapshotting should only be necessary for > certain applications that keep their files open, like DB servers, > Outlook and some FOSS mailsystems. And then only if these services > need to be kept running as close to 24/7 as possible, otherwise your > scripts can just close the server processes down until the backup is > complete and then bring them back up again. > I can't advise on the NTFS-specific extended attributes and > newfangled security stuff, but unless you're using software that > specifically leverages that MS-proprietary stuff, it shouldn't IMO > be an issue. > -- > For Developers, A Lot Can Happen In A Second. > Boundary is the first to Know...and Tell You. > Monitor Your Applications in Ultra-Fine Resolution. Try it FREE! > http://p.sf.net/sfu/Boundary-d2dvs2 > ___ > BackupPC-users mailing list > BackupPC-users@lists.sourceforge.net > List: https://lists.sourceforge.net/lists/listinfo/backuppc-users > Wiki: http://backuppc.wiki.sourceforge.net > Project: http://backuppc.sourceforge.net/ -- For Developers, A Lot Can Happen In A Second. Boundary is the first to Know...and Tell You. Monitor Your Applications in Ultra-Fine Resolution. Try it FREE! http://p.sf.net/sfu/Boundary-d2dvs2 ___ BackupPC-users mailing list BackupPC-users@lists.sourceforge.net List:https://lists.sourceforge.net/lists/listinfo/backuppc-users Wiki:http://backuppc.wiki.sourceforge.net Project: http://backuppc.sourceforge.net/
Re: [BackupPC-users] Restoring complete virtualized Windows-Servers / Saving MBR
Two completely separate backup schemes are needed here. One for full "cold-metal" restores of the boot/OS level stuff, and IMO this is best done with "imaging" style software, in your case specifically targeted for windoze/ntfs systems. These don't need to be done very frequently, as little is changing from day to day. BPC is not intended to provide for this kind of backup, especially regarding Windows. Many Linux sysadmins simply re-install their OS from automated scripts and then restore config files rather than bothering to fully image their boot/OS partitions, but Windows isn't suited to that approach. The type of backup is for working data, which requires the frequent full/incremental/archive that BPC is designed for. Details about the infrastructure "under" the filesystem are irrelevant to BPC, except when considering how to optimize performance when a small "backup window" becomes an issue. What you are doing with LVM snapshotting should only be necessary for certain applications that keep their files open, like DB servers, Outlook and some FOSS mailsystems. And then only if these services need to be kept running as close to 24/7 as possible, otherwise your scripts can just close the server processes down until the backup is complete and then bring them back up again. I can't advise on the NTFS-specific extended attributes and newfangled security stuff, but unless you're using software that specifically leverages that MS-proprietary stuff, it shouldn't IMO be an issue. -- For Developers, A Lot Can Happen In A Second. Boundary is the first to Know...and Tell You. Monitor Your Applications in Ultra-Fine Resolution. Try it FREE! http://p.sf.net/sfu/Boundary-d2dvs2 ___ BackupPC-users mailing list BackupPC-users@lists.sourceforge.net List:https://lists.sourceforge.net/lists/listinfo/backuppc-users Wiki:http://backuppc.wiki.sourceforge.net Project: http://backuppc.sourceforge.net/
Re: [BackupPC-users] Restoring complete virtualized Windows-Servers / Saving MBR
Hi, thank you for your response. At the moment I don't use disk images. Instead I use LVM volumes which are directly connected to my KVM-machines. There is a way like creating images of the LVM volumes with a image tool like partimage. These images would be compressed like 50GB in combined size, no problem to copy it to an external usb drive but much data to transfer over a VDSL50 internet connection. The point is that the customer wants me to backup the whole system (real-server including 2 VMs) over the network to a different location. So when the server gets unusable damaged for instance by beeing flooded with water because of a pipe-break, or the system gets stolen, I should be able to buy new hardware and get back to the state one day before the disaster occurred. I like the efficient way of file based backups backupPC uses, so I ask for experiences on that. But maybe I should search for a partition image tool that supports incremental backups. I only know of Acronis True Image but this is a commercial (and not cheap) way. Am 16.04.2012 um 09:28 schrieb hans...@gmail.com: > Yes, I see BackupPC as a solution for what I call "data archive" backups, as > opposed to "full host bare-metal". > > For the latter wrt physical machines I tend to do relatively infrequent > "image snapshots" of the boot and system partitions, keeping > frequently-changing "working data" on separate partitions, which are backed > up by BPC. > > I treat VM images as part of a third category, together with large media > files, either manually or via scripts, simply copying them to external > (esata) drives that get rotated offsite. > > For my use case, it would simply be impractical to have BPC keep so many > multiple copies of old versions of this third category, they're just too > large. The working data handled by the VMs is backed up by BPC (usually via a > central filer), but not the OS/boot partitions. > > -- > For Developers, A Lot Can Happen In A Second. > Boundary is the first to Know...and Tell You. > Monitor Your Applications in Ultra-Fine Resolution. Try it FREE! > http://p.sf.net/sfu/Boundary-d2dvs2 > ___ > BackupPC-users mailing list > BackupPC-users@lists.sourceforge.net > List:https://lists.sourceforge.net/lists/listinfo/backuppc-users > Wiki:http://backuppc.wiki.sourceforge.net > Project: http://backuppc.sourceforge.net/ -- For Developers, A Lot Can Happen In A Second. Boundary is the first to Know...and Tell You. Monitor Your Applications in Ultra-Fine Resolution. Try it FREE! http://p.sf.net/sfu/Boundary-d2dvs2 ___ BackupPC-users mailing list BackupPC-users@lists.sourceforge.net List:https://lists.sourceforge.net/lists/listinfo/backuppc-users Wiki:http://backuppc.wiki.sourceforge.net Project: http://backuppc.sourceforge.net/
Re: [BackupPC-users] Restoring complete virtualized Windows-Servers / Saving MBR
Hi Jim, I think I was not able to make myself perfectly clear: I don't use virtual disk images. I use LVM-Volumes instead (you can think of them as partitions that are presented as block devices to ma virtual machines). One thing you stated about doing virtual disk image backups is that they're huge. As soon as anything in the virtual machine changes, the complete image needs to be stored again to keep the image backup in sync. That's what I want to avoid by using LVM volumes: I can create a R/O snapshot of them and mount them into the filesystem just like an external harddrive connected to the system. From there I can do a file-based backups which is way more space efficient. As I don't have any experiences in restoring a virtual machine from a file-based backup (I think of possible permission and special attribute problems from the NTFS-filesystem/driver, missing boot-record etc.) I ask if someone can tell me about that or if this is not a working backup solution at all. Thank you for your response, Andreas Piening Am 16.04.2012 um 02:56 schrieb Jim Kyle: > On Sunday, April 15, 2012, at 7:37:20 PM, Andreas Piening wrote: > > =>> I need to be able to completely restore the system including the > =>> virtual machines if the machine gets lost, or damaged to a > =>> non-repairable state > > If you do a full backup of the host, including all the KVM configuration > and VM files together with the virtual disk files themselves, then a full > restore should automagically take care of the virtual machines. They would > not need to be backed up separately. You should also be able to restore > their data by simply restoring just the virtual disk files, included in the > full host backup. > > I don't do this myself, though, to back up my VirtualBox virtual machines, > because my backup media isn't large enough to handle full host backups. > Instead, I simply copy the virtual disk files individually to one of > several external drives. This has worked to give me full backup on the VMs > on the few occasions that I've needed it. > > -- > Jim Kyle > mailto: j...@jimkyle.com > > > -- > For Developers, A Lot Can Happen In A Second. > Boundary is the first to Know...and Tell You. > Monitor Your Applications in Ultra-Fine Resolution. Try it FREE! > http://p.sf.net/sfu/Boundary-d2dvs2 > ___ > BackupPC-users mailing list > BackupPC-users@lists.sourceforge.net > List:https://lists.sourceforge.net/lists/listinfo/backuppc-users > Wiki:http://backuppc.wiki.sourceforge.net > Project: http://backuppc.sourceforge.net/ -- For Developers, A Lot Can Happen In A Second. Boundary is the first to Know...and Tell You. Monitor Your Applications in Ultra-Fine Resolution. Try it FREE! http://p.sf.net/sfu/Boundary-d2dvs2 ___ BackupPC-users mailing list BackupPC-users@lists.sourceforge.net List:https://lists.sourceforge.net/lists/listinfo/backuppc-users Wiki:http://backuppc.wiki.sourceforge.net Project: http://backuppc.sourceforge.net/
Re: [BackupPC-users] Restoring complete virtualized Windows-Servers / Saving MBR
Yes, I see BackupPC as a solution for what I call "data archive" backups, as opposed to "full host bare-metal". For the latter wrt physical machines I tend to do relatively infrequent "image snapshots" of the boot and system partitions, keeping frequently-changing "working data" on separate partitions, which are backed up by BPC. I treat VM images as part of a third category, together with large media files, either manually or via scripts, simply copying them to external (esata) drives that get rotated offsite. For my use case, it would simply be impractical to have BPC keep so many multiple copies of old versions of this third category, they're just too large. The working data handled by the VMs is backed up by BPC (usually via a central filer), but not the OS/boot partitions. -- For Developers, A Lot Can Happen In A Second. Boundary is the first to Know...and Tell You. Monitor Your Applications in Ultra-Fine Resolution. Try it FREE! http://p.sf.net/sfu/Boundary-d2dvs2 ___ BackupPC-users mailing list BackupPC-users@lists.sourceforge.net List:https://lists.sourceforge.net/lists/listinfo/backuppc-users Wiki:http://backuppc.wiki.sourceforge.net Project: http://backuppc.sourceforge.net/
Re: [BackupPC-users] Restoring complete virtualized Windows-Servers / Saving MBR
On Sunday, April 15, 2012, at 7:37:20 PM, Andreas Piening wrote: =>> I need to be able to completely restore the system including the =>> virtual machines if the machine gets lost, or damaged to a =>> non-repairable state If you do a full backup of the host, including all the KVM configuration and VM files together with the virtual disk files themselves, then a full restore should automagically take care of the virtual machines. They would not need to be backed up separately. You should also be able to restore their data by simply restoring just the virtual disk files, included in the full host backup. I don't do this myself, though, to back up my VirtualBox virtual machines, because my backup media isn't large enough to handle full host backups. Instead, I simply copy the virtual disk files individually to one of several external drives. This has worked to give me full backup on the VMs on the few occasions that I've needed it. -- Jim Kyle mailto: j...@jimkyle.com -- For Developers, A Lot Can Happen In A Second. Boundary is the first to Know...and Tell You. Monitor Your Applications in Ultra-Fine Resolution. Try it FREE! http://p.sf.net/sfu/Boundary-d2dvs2 ___ BackupPC-users mailing list BackupPC-users@lists.sourceforge.net List:https://lists.sourceforge.net/lists/listinfo/backuppc-users Wiki:http://backuppc.wiki.sourceforge.net Project: http://backuppc.sourceforge.net/
[BackupPC-users] Restoring complete virtualized Windows-Servers / Saving MBR
Hello BackupPC-Users, I am planing to use BackupPC to backup a server with 2 virtual machines over the network and I have a few questions on that. Following scenario: => The server is linux-based (ssh/rsync available) => There are 2 virtual machines (KVM) running Windows Server 2008 R2 64 bit => Each virtual server uses 2 LVM volumes as their system disks => I need to be able to completely restore the system including the virtual machines if the machine gets lost, or damaged to a non-repairable state => I can backup the linux system via rsync. I can create R/O snapshots for the LVM-volumes used by the virtual machines (KVM) and backup them. This way I can do the backup without stopping the machines. Moreover I can use rsync instead of SMB which supports compression on the network and has better performance, right? Here are my questions: => If i want to restore the virtualized Windows Server 2008 machines, is it enough to mount the lvm-volumes RW and copy all files back via BackupPC/rsync? Do I need an additional MBR-backup to get the volume bootable after restore? May I run into file permission issues or problems with special attributes, because the windows-servers are using NTFS as their filesystems. How can I handle this? => BackupPC normally allows me to download files or folders from the backup-set, or restore them to the same destination where they're backuped from. Will I be able to restore via rsync to another location of my choice (for instance when I want to re-install the system on new hardware)? If there is some documentation which gives me hints on that, feel free to point me on this. Thank you in advance, Andreas Piening -- For Developers, A Lot Can Happen In A Second. Boundary is the first to Know...and Tell You. Monitor Your Applications in Ultra-Fine Resolution. Try it FREE! http://p.sf.net/sfu/Boundary-d2dvs2 ___ BackupPC-users mailing list BackupPC-users@lists.sourceforge.net List:https://lists.sourceforge.net/lists/listinfo/backuppc-users Wiki:http://backuppc.wiki.sourceforge.net Project: http://backuppc.sourceforge.net/
