[BackupPC-users] Backuppc and KVM virtual machine
I decided to created a VM to move my backuppc installation. New to KVM - just created a couple VM guests over Ubuntu Server 12.04. Everything was running fine for several days with a fresh backuppc pool. Then I came home to 2 stopped guest VM's. I tried to restart. One won't since the virtual file system for it is gone -completely (backuppc VM). The error when I try and start the backuppc VM is: Error starting domain: Unable to allow access for disk path /./..qcow2: No such file or directory I check on the host file system and there is no longer any file by that name there... Nothing. As best I can estimate the backuppc VM was to about 20% of disk capacity when it crashed and disappeared. The second VM is fine. Both VM's used the qcow2 file format over an ext4 file system. Both VM images are on the same drive and mount. Seemingly one has just completely disappeared. Am I wrong to use a VM server to store a complex file system like backuppc uses? Should I build the VM to only hold the OS side of things - then mount and external file system to hold my backuppc file system. I plan to store about 2TB of data is this VM. Based on my experiments, it seems I'd be better off with a physical machine for this type of environment. -- Rapidly troubleshoot problems before they affect your business. Most IT organizations don't have a clear picture of how application performance affects their revenue. With AppDynamics, you get 100% visibility into your Java,.NET, & PHP application. Start your 15-day FREE TRIAL of AppDynamics Pro! http://pubads.g.doubleclick.net/gampad/clk?id=84349831&iu=/4140/ostg.clktrk ___ BackupPC-users mailing list BackupPC-users@lists.sourceforge.net List:https://lists.sourceforge.net/lists/listinfo/backuppc-users Wiki:http://backuppc.wiki.sourceforge.net Project: http://backuppc.sourceforge.net/
Re: [BackupPC-users] Backuppc and KVM virtual machine
Hi David, I am also using KVM and VM for the backuppc running on Ubuntu 12.04. It can run without issues but based on my observatio, you need to roughly allocate 1 core for 1 client that requires backup. For example if you have 10 clients then its better to allocate 10 cores. I got this through the NMON monitoring when multiple jobs are running at the same time. Any other thoughts regading this is welcomed. From: David Trebacz To: backuppc-users@lists.sourceforge.net Sent: Friday, December 27, 2013 10:54 AM Subject: [BackupPC-users] Backuppc and KVM virtual machine I decided to created a VM to move my backuppc installation. New to KVM - just created a couple VM guests over Ubuntu Server 12.04. Everything was running fine for several days with a fresh backuppc pool. Then I came home to 2 stopped guest VM's. I tried to restart. One won't since the virtual file system for it is gone -completely (backuppc VM). The error when I try and start the backuppc VM is: Error starting domain: Unable to allow access for disk path /./..qcow2: No such file or directory I check on the host file system and there is no longer any file by that name there... Nothing. As best I can estimate the backuppc VM was to about 20% of disk capacity when it crashed and disappeared. The second VM is fine. Both VM's used the qcow2 file format over an ext4 file system. Both VM images are on the same drive and mount. Seemingly one has just completely disappeared. Am I wrong to use a VM server to store a complex file system like backuppc uses? Should I build the VM to only hold the OS side of things - then mount and external file system to hold my backuppc file system. I plan to store about 2TB of data is this VM. Based on my experiments, it seems I'd be better off with a physical machine for this type of environment. -- Rapidly troubleshoot problems before they affect your business. Most IT organizations don't have a clear picture of how application performance affects their revenue. With AppDynamics, you get 100% visibility into your Java,.NET, & PHP application. Start your 15-day FREE TRIAL of AppDynamics Pro! http://pubads.g.doubleclick.net/gampad/clk?id=84349831&iu=/4140/ostg.clktrk ___ BackupPC-users mailing list BackupPC-users@lists.sourceforge.net List: https://lists.sourceforge.net/lists/listinfo/backuppc-users Wiki: http://backuppc.wiki.sourceforge.net Project: http://backuppc.sourceforge.net/-- Rapidly troubleshoot problems before they affect your business. Most IT organizations don't have a clear picture of how application performance affects their revenue. With AppDynamics, you get 100% visibility into your Java,.NET, & PHP application. Start your 15-day FREE TRIAL of AppDynamics Pro! http://pubads.g.doubleclick.net/gampad/clk?id=84349831&iu=/4140/ostg.clktrk___ BackupPC-users mailing list BackupPC-users@lists.sourceforge.net List:https://lists.sourceforge.net/lists/listinfo/backuppc-users Wiki:http://backuppc.wiki.sourceforge.net Project: http://backuppc.sourceforge.net/
Re: [BackupPC-users] Backuppc and KVM virtual machine
On 13-12-27 12:28 AM, Prem wrote: > Hi David, > > I am also using KVM and VM for the backuppc running on Ubuntu 12.04. > It can run without issues but based on my observatio, you need to > roughly allocate 1 core for 1 client that requires backup. For example > if you have 10 clients then its better to allocate 10 cores. > > I got this through the NMON monitoring when multiple jobs are running > at the same time. > You can control CPU/disk IO by limiting the number of jobs that run in parallel. You'll also find better efficiency by using OpenVZ rather than KVM. For a very nice KVM/OpenVZ interface, check out proxmox (http://www.proxmox.com/) -- Looking for (employment|contract) work in the Internet industry, preferably working remotely. Building / Supporting the net since 2400 baud was the hot thing. Ask for a resume! ispbuil...@gmail.com -- Rapidly troubleshoot problems before they affect your business. Most IT organizations don't have a clear picture of how application performance affects their revenue. With AppDynamics, you get 100% visibility into your Java,.NET, & PHP application. Start your 15-day FREE TRIAL of AppDynamics Pro! http://pubads.g.doubleclick.net/gampad/clk?id=84349831&iu=/4140/ostg.clktrk___ BackupPC-users mailing list BackupPC-users@lists.sourceforge.net List:https://lists.sourceforge.net/lists/listinfo/backuppc-users Wiki:http://backuppc.wiki.sourceforge.net Project: http://backuppc.sourceforge.net/
Re: [BackupPC-users] Backuppc and KVM virtual machine
The VM isn't an issue (my BackupPC has run in a KVM guest for over two years), but using "file-based" storage (a filesystem inside a file stored on another filesystem) is extremely inefficient (we recently ran benchmarks on file versus raw device... the performance is awful in comparison). Use a raw device (raw disk, LVM volume, etc) with the virtio driver. Your OS and backup storage can reside on the same raw device, or they can be separate raw devices, or if you insist, the OS can be virtual file storage. We just use raw devices (LVM logical volumes) for everything. My disk configuration looks like this (virsh dumpxml ): "/dev/diskbackup/diskbackup" is a logical volume named "diskbackup" in a volume group named "diskbackup". (I didn't configure this... traditionally, this would be "/dev/vgdiskbackup/lvdiskbackup" or such.) -- Carl D Cravens (ccrav...@excelii.com), Ext 228 (620.327.1228) Lead System Architect -- Rapidly troubleshoot problems before they affect your business. Most IT organizations don't have a clear picture of how application performance affects their revenue. With AppDynamics, you get 100% visibility into your Java,.NET, & PHP application. Start your 15-day FREE TRIAL of AppDynamics Pro! http://pubads.g.doubleclick.net/gampad/clk?id=84349831&iu=/4140/ostg.clktrk ___ BackupPC-users mailing list BackupPC-users@lists.sourceforge.net List:https://lists.sourceforge.net/lists/listinfo/backuppc-users Wiki:http://backuppc.wiki.sourceforge.net Project: http://backuppc.sourceforge.net/
Re: [BackupPC-users] Backuppc and KVM virtual machine
Le vendredi 03 janvier 2014 à 09:46 -0600, Carl Cravens a écrit : > The VM isn't an issue (my BackupPC has run in a KVM guest for over two > years), but using "file-based" storage (a filesystem inside a file stored on > another filesystem) is extremely inefficient (we recently ran benchmarks on > file versus raw device... the performance is awful in comparison). > > Use a raw device (raw disk, LVM volume, etc) with the virtio driver. Your OS > and backup storage can reside on the same raw device, or they can be separate > raw devices, or if you insist, the OS can be virtual file storage. We just > use raw devices (LVM logical volumes) for everything. > > My disk configuration looks like this (virsh dumpxml ): > > > > > > > function='0x0'/> > > > "/dev/diskbackup/diskbackup" is a logical volume named "diskbackup" in a > volume group named "diskbackup". (I didn't configure this... traditionally, > this would be "/dev/vgdiskbackup/lvdiskbackup" or such.) > Morning, to boost I/O performance you can also activate the memory cache over the virtual disk into KVM this configuration give you performance like native one (without virtualisation) there is a limitation: - live migration between kvm hosts is not possible as i know. - As every time cache is used in case of power outage there is a possibility of lost data mna. -- Rapidly troubleshoot problems before they affect your business. Most IT organizations don't have a clear picture of how application performance affects their revenue. With AppDynamics, you get 100% visibility into your Java,.NET, & PHP application. Start your 15-day FREE TRIAL of AppDynamics Pro! http://pubads.g.doubleclick.net/gampad/clk?id=84349831&iu=/4140/ostg.clktrk ___ BackupPC-users mailing list BackupPC-users@lists.sourceforge.net List:https://lists.sourceforge.net/lists/listinfo/backuppc-users Wiki:http://backuppc.wiki.sourceforge.net Project: http://backuppc.sourceforge.net/
Re: [BackupPC-users] Backuppc and KVM virtual machine
Thanks Carl. This was the direction/experience I was looking for. I always felt that running a complex file system like backuppc's on a virtualized file system would have additional performance implications. I went ahead and tried it and had issues. Using the virtio driver just makes more sense to me. It also allows for better portability of the VM for backuppc. David Trebacz Email: da...@trebacz.com Tel: 815-344-9068 Mobile: 815-529-2638 On 01/03/2014 09:46 AM, Carl Cravens wrote: > The VM isn't an issue (my BackupPC has run in a KVM guest for over two > years), but using "file-based" storage (a filesystem inside a file stored on > another filesystem) is extremely inefficient (we recently ran benchmarks on > file versus raw device... the performance is awful in comparison). > > Use a raw device (raw disk, LVM volume, etc) with the virtio driver. Your OS > and backup storage can reside on the same raw device, or they can be separate > raw devices, or if you insist, the OS can be virtual file storage. We just > use raw devices (LVM logical volumes) for everything. > > My disk configuration looks like this (virsh dumpxml ): > > > > > > > function='0x0'/> > > > "/dev/diskbackup/diskbackup" is a logical volume named "diskbackup" in a > volume group named "diskbackup". (I didn't configure this... traditionally, > this would be "/dev/vgdiskbackup/lvdiskbackup" or such.) > -- Rapidly troubleshoot problems before they affect your business. Most IT organizations don't have a clear picture of how application performance affects their revenue. With AppDynamics, you get 100% visibility into your Java,.NET, & PHP application. Start your 15-day FREE TRIAL of AppDynamics Pro! http://pubads.g.doubleclick.net/gampad/clk?id=84349831&iu=/4140/ostg.clktrk ___ BackupPC-users mailing list BackupPC-users@lists.sourceforge.net List:https://lists.sourceforge.net/lists/listinfo/backuppc-users Wiki:http://backuppc.wiki.sourceforge.net Project: http://backuppc.sourceforge.net/
