Re: [Bug 1025244] Re: qcow2 image increasing disk size above the virtual limit
@Mario, the external snapshots have apparently been around a long time. The ability to create external snapshots from running vms is newer, but it appears to exist evn in qemu-kvm 1.0. So all versions in Debian and Ubuntu should support them. http://wiki.qemu.org/Features/Snapshots#Snapshot_command_flow -- You received this bug notification because you are a member of Ubuntu Server Team, which is subscribed to qemu-kvm in Ubuntu. https://bugs.launchpad.net/bugs/1025244 Title: qcow2 image increasing disk size above the virtual limit To manage notifications about this bug go to: https://bugs.launchpad.net/qemu/+bug/1025244/+subscriptions -- Ubuntu-server-bugs mailing list Ubuntu-server-bugs@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-server-bugs
[Bug 1025244] Re: qcow2 image increasing disk size above the virtual limit
@Mario, in theory an image that should be taking up 30 GB with four snapshots should be taking up at most about 150 GB, of course. Now the question is what you mean by should be taking up 30 GB and by is taking 600+ GB. For the latter, did you query the file length (ls -l) or the actual size (qemu-img info, disk size)? For the former, if you have a virtual disk size of 1 TB and the guest reports 30 GB are used, that doesn't mean that qemu knows that only 30 GB are used. If you delete a file in the guest, it will report less space being used; however, qemu doesn't know about that unless the guest bothers to discard the now unused sectors. If it doesn't (and I don't see a reason why a guest should discard sectors on an HDD), the guest will just remove the file metadata but the data will stay there (and may be overwritten later by the guest when creating new files etc.). qemu has no idea that that data is now unused, therefore it must treat those sectors as being in use. If your image indeed has a virtual disk size of 30 GB, has four snapshots, is clean (qemu-img check) and does take up 600+ GB of actual disk space, that should indeed not be happening (unless there's some case I forgot to consider). -- You received this bug notification because you are a member of Ubuntu Server Team, which is subscribed to qemu-kvm in Ubuntu. https://bugs.launchpad.net/bugs/1025244 Title: qcow2 image increasing disk size above the virtual limit To manage notifications about this bug go to: https://bugs.launchpad.net/qemu/+bug/1025244/+subscriptions -- Ubuntu-server-bugs mailing list Ubuntu-server-bugs@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-server-bugs
[Bug 1025244] Re: qcow2 image increasing disk size above the virtual limit
@serge, what version would I need to upgrade to be able to use the external snapshots? that sounds like it would solve my problems -- You received this bug notification because you are a member of Ubuntu Server Team, which is subscribed to qemu-kvm in Ubuntu. https://bugs.launchpad.net/bugs/1025244 Title: qcow2 image increasing disk size above the virtual limit To manage notifications about this bug go to: https://bugs.launchpad.net/qemu/+bug/1025244/+subscriptions -- Ubuntu-server-bugs mailing list Ubuntu-server-bugs@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-server-bugs
[Bug 1025244] Re: qcow2 image increasing disk size above the virtual limit
Changing priority given workarounds. ** Changed in: qemu-kvm (Ubuntu) Importance: High = Low -- You received this bug notification because you are a member of Ubuntu Server Team, which is subscribed to qemu-kvm in Ubuntu. https://bugs.launchpad.net/bugs/1025244 Title: qcow2 image increasing disk size above the virtual limit To manage notifications about this bug go to: https://bugs.launchpad.net/qemu/+bug/1025244/+subscriptions -- Ubuntu-server-bugs mailing list Ubuntu-server-bugs@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-server-bugs
[Bug 1025244] Re: qcow2 image increasing disk size above the virtual limit
@michael, so you do that once, after some time the machine keeps growing, and growing and growing... and you have to redo that every so often... I have a machine that should be taking up 30 gb yet is taking 600+ GB with 4 snapshots... but yeah... I'll just plug in another 1tb hard drive so that i can free up the space only for it to happen again in a near future... Seems a great workaround! -- You received this bug notification because you are a member of Ubuntu Server Team, which is subscribed to qemu-kvm in Ubuntu. https://bugs.launchpad.net/bugs/1025244 Title: qcow2 image increasing disk size above the virtual limit To manage notifications about this bug go to: https://bugs.launchpad.net/qemu/+bug/1025244/+subscriptions -- Ubuntu-server-bugs mailing list Ubuntu-server-bugs@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-server-bugs
Re: [Bug 1025244] Re: qcow2 image increasing disk size above the virtual limit
For the record, the workaround is deleting old snapshots in shutdown mode as per comment #14. Upstream has moved toward external snapshots as the way forward, so while I don't argue that this is a bug, it seems unlikely to receive a fix from upstream. -- You received this bug notification because you are a member of Ubuntu Server Team, which is subscribed to qemu-kvm in Ubuntu. https://bugs.launchpad.net/bugs/1025244 Title: qcow2 image increasing disk size above the virtual limit To manage notifications about this bug go to: https://bugs.launchpad.net/qemu/+bug/1025244/+subscriptions -- Ubuntu-server-bugs mailing list Ubuntu-server-bugs@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-server-bugs
[Bug 1025244] Re: qcow2 image increasing disk size above the virtual limit
Looking at what? At the lack of problems as comment #14 says? -- You received this bug notification because you are a member of Ubuntu Server Team, which is subscribed to qemu-kvm in Ubuntu. https://bugs.launchpad.net/bugs/1025244 Title: qcow2 image increasing disk size above the virtual limit To manage notifications about this bug go to: https://bugs.launchpad.net/qemu/+bug/1025244/+subscriptions -- Ubuntu-server-bugs mailing list Ubuntu-server-bugs@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-server-bugs
[Bug 1025244] Re: qcow2 image increasing disk size above the virtual limit
Is anyone even looking at this? been years and the problem still persists! -- You received this bug notification because you are a member of Ubuntu Server Team, which is subscribed to qemu-kvm in Ubuntu. https://bugs.launchpad.net/bugs/1025244 Title: qcow2 image increasing disk size above the virtual limit To manage notifications about this bug go to: https://bugs.launchpad.net/qemu/+bug/1025244/+subscriptions -- Ubuntu-server-bugs mailing list Ubuntu-server-bugs@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-server-bugs
Re: [Qemu-devel] [Bug 1025244] Re: qcow2 image increasing disk size above the virtual limit
On Tue, Dec 18, 2012 at 10:18:20AM -, Andy Menzel wrote: Any solution right now? I have a similar problem like Todor Andreev; Our daily backup of some virtual machines (qcow2) looks like that: 1. shutdown the VM 2. create a snapshot via: qemu-img snapshot -c nameofsnapshot... 3. boot the VM 4. backup the snapshot to another virtual disk via: qemu-img convert -f qcow2 -O qcow2 -s nameofsnapshot... 5. DELETE the snapshot from VM via: qemu-img snapshot -d nameofsnapshot... It's not safe to modify the qcow2 file while the guest is running. This means Step 5 is not really safe and could result in an inconsistent image. This may also be causing the problem: the QEMU process has a variable with the next free cluster index. Since Step 5 runs as a separate process it does not update the QEMU process' next free cluster index variable. QEMU doesn't know that there are now free clusters within the image file because you updated the file behind QEMU's back - the result is that it grows the file. Please try deleting the last backup snapshot between Step 1 and Step 2. This way you'll free the space while QEMU isn't accessing the image file. When you boot up the image file again QEMU should reuse the freed clusters. Stefan -- You received this bug notification because you are a member of Ubuntu Server Team, which is subscribed to qemu-kvm in Ubuntu. https://bugs.launchpad.net/bugs/1025244 Title: qcow2 image increasing disk size above the virtual limit To manage notifications about this bug go to: https://bugs.launchpad.net/qemu/+bug/1025244/+subscriptions -- Ubuntu-server-bugs mailing list Ubuntu-server-bugs@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-server-bugs
Re: [Qemu-devel] [Bug 1025244] Re: qcow2 image increasing disk size above the virtual limit
On 01/02/2013 08:50 AM, Stefan Hajnoczi wrote: On Tue, Dec 18, 2012 at 10:18:20AM -, Andy Menzel wrote: Any solution right now? I have a similar problem like Todor Andreev; Our daily backup of some virtual machines (qcow2) looks like that: 1. shutdown the VM 2. create a snapshot via: qemu-img snapshot -c nameofsnapshot... 3. boot the VM 4. backup the snapshot to another virtual disk via: qemu-img convert -f qcow2 -O qcow2 -s nameofsnapshot... 5. DELETE the snapshot from VM via: qemu-img snapshot -d nameofsnapshot... It's not safe to modify the qcow2 file while the guest is running. This means Step 5 is not really safe and could result in an inconsistent image. This may also be causing the problem: the QEMU process has a variable with the next free cluster index. Since Step 5 runs as a separate process it does not update the QEMU process' next free cluster index variable. QEMU doesn't know that there are now free clusters within the image file because you updated the file behind QEMU's back - the result is that it grows the file. Please try deleting the last backup snapshot between Step 1 and Step 2. This way you'll free the space while QEMU isn't accessing the image file. When you boot up the image file again QEMU should reuse the freed clusters. You might also want to try modifying step 5 to use the HMP delvm monitor command from within the running qemu rather than going behind qemu's back with a qemu-img invocation. That's how libvirt deletes internal snapshots from a running qemu. Also, there are patches currently under review that are talking about creating a QMP counterpart to the delvm monitor command. -- Eric Blake eblake redhat com+1-919-301-3266 Libvirt virtualization library http://libvirt.org -- You received this bug notification because you are a member of Ubuntu Server Team, which is subscribed to qemu-kvm in Ubuntu. https://bugs.launchpad.net/bugs/1025244 Title: qcow2 image increasing disk size above the virtual limit To manage notifications about this bug go to: https://bugs.launchpad.net/qemu/+bug/1025244/+subscriptions -- Ubuntu-server-bugs mailing list Ubuntu-server-bugs@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-server-bugs
[Bug 1025244] Re: qcow2 image increasing disk size above the virtual limit
I don't know of any qcow2-based workaround. Is anyone actively working on fixing the qcow2 code? In particular, the fact that after removing snapshots, un-used blocks are not reclaimed and disk size is never reduced? One possible workaround (the one I would use) would be to use lvm-based snapshotting instead. -- You received this bug notification because you are a member of Ubuntu Server Team, which is subscribed to qemu-kvm in Ubuntu. https://bugs.launchpad.net/bugs/1025244 Title: qcow2 image increasing disk size above the virtual limit To manage notifications about this bug go to: https://bugs.launchpad.net/qemu/+bug/1025244/+subscriptions -- Ubuntu-server-bugs mailing list Ubuntu-server-bugs@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-server-bugs
[Bug 1025244] Re: qcow2 image increasing disk size above the virtual limit
Any solution right now? I have a similar problem like Todor Andreev; Our daily backup of some virtual machines (qcow2) looks like that: 1. shutdown the VM 2. create a snapshot via: qemu-img snapshot -c nameofsnapshot... 3. boot the VM 4. backup the snapshot to another virtual disk via: qemu-img convert -f qcow2 -O qcow2 -s nameofsnapshot... 5. DELETE the snapshot from VM via: qemu-img snapshot -d nameofsnapshot... But the problem is, that our original VM-size growing steadily (although few changes were made) ?! -- You received this bug notification because you are a member of Ubuntu Server Team, which is subscribed to qemu-kvm in Ubuntu. https://bugs.launchpad.net/bugs/1025244 Title: qcow2 image increasing disk size above the virtual limit To manage notifications about this bug go to: https://bugs.launchpad.net/qemu/+bug/1025244/+subscriptions -- Ubuntu-server-bugs mailing list Ubuntu-server-bugs@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-server-bugs