Re: [Qemu-devel] [PATCH v7 0/4] Add shrink image for qcow2
On 16.09.2017 17:56, Max Reitz wrote: On 2017-08-22 01:31, John Snow wrote: On 08/17/2017 05:15 AM, Pavel Butsykin wrote: This patch add shrinking of the image file for qcow2. As a result, this allows us to reduce the virtual image size and free up space on the disk without copying the image. Image can be fragmented and shrink is done by punching holes in the image file. # ./qemu-img create -f qcow2 image.qcow2 4G Formatting 'image.qcow2', fmt=qcow2 size=4294967296 encryption=off cluster_size=65536 lazy_refcounts=off refcount_bits=16 # ./qemu-io -c "write -P 0x22 0 1G" image.qcow2 wrote 1073741824/1073741824 bytes at offset 0 1 GiB, 1 ops; 0:00:04.59 (222.886 MiB/sec and 0.2177 ops/sec) # ./qemu-img resize image.qcow2 512M warning: qemu-img: Shrinking an image will delete all data beyond the shrunken image's end. Before performing such an operation, make sure there is no important data there. error: qemu-img: Use the --shrink option to perform a shrink operation. # ./qemu-img resize --shrink image.qcow2 128M Image resized. # ./qemu-img info image.qcow2 image: image.qcow2 file format: qcow2 virtual size: 128M (134217728 bytes) disk size: 128M cluster_size: 65536 Format specific information: compat: 1.1 lazy refcounts: false refcount bits: 16 corrupt: false # du -h image.qcow2 129Mimage.qcow2 It looks sane to me, and already has a full set of R-Bs from Max. Are we waiting for Kevin? Now I found that I was waiting for my own R-b for patch 3. I hadn't yet reviewed that patch in its current state (I had reviewed it for v5, but it has changed quite a bit in v6) I decided that small changes are not considered :) I'm sorry for the confusion with the R-b. Max
Re: [Qemu-devel] [PATCH v7 0/4] Add shrink image for qcow2
On 2017-08-22 01:31, John Snow wrote: > > > On 08/17/2017 05:15 AM, Pavel Butsykin wrote: >> This patch add shrinking of the image file for qcow2. As a result, this >> allows >> us to reduce the virtual image size and free up space on the disk without >> copying the image. Image can be fragmented and shrink is done by punching >> holes >> in the image file. >> >> # ./qemu-img create -f qcow2 image.qcow2 4G >> Formatting 'image.qcow2', fmt=qcow2 size=4294967296 encryption=off >> cluster_size=65536 lazy_refcounts=off refcount_bits=16 >> >> # ./qemu-io -c "write -P 0x22 0 1G" image.qcow2 >> wrote 1073741824/1073741824 bytes at offset 0 >> 1 GiB, 1 ops; 0:00:04.59 (222.886 MiB/sec and 0.2177 ops/sec) >> >> # ./qemu-img resize image.qcow2 512M >> warning: qemu-img: Shrinking an image will delete all data beyond the >> shrunken image's end. Before performing such an operation, make sure there >> is no important data there. >> error: qemu-img: Use the --shrink option to perform a shrink operation. >> >> # ./qemu-img resize --shrink image.qcow2 128M >> Image resized. >> >> # ./qemu-img info image.qcow2 >> image: image.qcow2 >> file format: qcow2 >> virtual size: 128M (134217728 bytes) >> disk size: 128M >> cluster_size: 65536 >> Format specific information: >> compat: 1.1 >> lazy refcounts: false >> refcount bits: 16 >> corrupt: false >> >> # du -h image.qcow2 >> 129Mimage.qcow2 >> > > It looks sane to me, and already has a full set of R-Bs from Max. Are we > waiting for Kevin? Now I found that I was waiting for my own R-b for patch 3. I hadn't yet reviewed that patch in its current state (I had reviewed it for v5, but it has changed quite a bit in v6). Max signature.asc Description: OpenPGP digital signature
Re: [Qemu-devel] [PATCH v7 0/4] Add shrink image for qcow2
On 17.08.2017 12:15, Pavel Butsykin wrote: This patch add shrinking of the image file for qcow2. As a result, this allows us to reduce the virtual image size and free up space on the disk without copying the image. Image can be fragmented and shrink is done by punching holes in the image file. # ./qemu-img create -f qcow2 image.qcow2 4G Formatting 'image.qcow2', fmt=qcow2 size=4294967296 encryption=off cluster_size=65536 lazy_refcounts=off refcount_bits=16 # ./qemu-io -c "write -P 0x22 0 1G" image.qcow2 wrote 1073741824/1073741824 bytes at offset 0 1 GiB, 1 ops; 0:00:04.59 (222.886 MiB/sec and 0.2177 ops/sec) # ./qemu-img resize image.qcow2 512M warning: qemu-img: Shrinking an image will delete all data beyond the shrunken image's end. Before performing such an operation, make sure there is no important data there. error: qemu-img: Use the --shrink option to perform a shrink operation. # ./qemu-img resize --shrink image.qcow2 128M Image resized. # ./qemu-img info image.qcow2 image: image.qcow2 file format: qcow2 virtual size: 128M (134217728 bytes) disk size: 128M cluster_size: 65536 Format specific information: compat: 1.1 lazy refcounts: false refcount bits: 16 corrupt: false # du -h image.qcow2 129Mimage.qcow2 ping Hi guys, I would like to ask is there any plans to take this small patch series? :) And thanks for the review.
Re: [Qemu-devel] [PATCH v7 0/4] Add shrink image for qcow2
On 2017-08-22 01:31, John Snow wrote: > > > On 08/17/2017 05:15 AM, Pavel Butsykin wrote: >> This patch add shrinking of the image file for qcow2. As a result, this >> allows >> us to reduce the virtual image size and free up space on the disk without >> copying the image. Image can be fragmented and shrink is done by punching >> holes >> in the image file. >> >> # ./qemu-img create -f qcow2 image.qcow2 4G >> Formatting 'image.qcow2', fmt=qcow2 size=4294967296 encryption=off >> cluster_size=65536 lazy_refcounts=off refcount_bits=16 >> >> # ./qemu-io -c "write -P 0x22 0 1G" image.qcow2 >> wrote 1073741824/1073741824 bytes at offset 0 >> 1 GiB, 1 ops; 0:00:04.59 (222.886 MiB/sec and 0.2177 ops/sec) >> >> # ./qemu-img resize image.qcow2 512M >> warning: qemu-img: Shrinking an image will delete all data beyond the >> shrunken image's end. Before performing such an operation, make sure there >> is no important data there. >> error: qemu-img: Use the --shrink option to perform a shrink operation. >> >> # ./qemu-img resize --shrink image.qcow2 128M >> Image resized. >> >> # ./qemu-img info image.qcow2 >> image: image.qcow2 >> file format: qcow2 >> virtual size: 128M (134217728 bytes) >> disk size: 128M >> cluster_size: 65536 >> Format specific information: >> compat: 1.1 >> lazy refcounts: false >> refcount bits: 16 >> corrupt: false >> >> # du -h image.qcow2 >> 129Mimage.qcow2 >> > > It looks sane to me, and already has a full set of R-Bs from Max. Are we > waiting for Kevin? We were waiting for Kevin to handle 2.10 patches so he could go into PTO and for me to come back from PTO. ;-) I'm still sifting through my inbox... When I'm done, I'll take a look, but I have no idea how long that might last. (Since 2.10 isn't out yet, there might be more pressing issues still...? I don't quite now yet, to be honest...) > It looks like in Vladimir's series we sidestepped the problem of what > happens if we resize with persistent bitmaps: we deny the operation, > regardless of if we are shrinking, growing, or have bitmaps that are > read-only, frozen, or what-have-you. > > It shouldn't be too hard to add soon, but this is fine for now. > (I think for adding it we just need to make sure the bitmaps aren't > frozen and are not read-only, and the implementation bitmap structure > can already handle truncation in either direction just fine.) > > Anyway; > > Reviewed-by: John Snow Thanks, in any case. :-) Max signature.asc Description: OpenPGP digital signature
Re: [Qemu-devel] [PATCH v7 0/4] Add shrink image for qcow2
On 08/17/2017 05:15 AM, Pavel Butsykin wrote: > This patch add shrinking of the image file for qcow2. As a result, this allows > us to reduce the virtual image size and free up space on the disk without > copying the image. Image can be fragmented and shrink is done by punching > holes > in the image file. > > # ./qemu-img create -f qcow2 image.qcow2 4G > Formatting 'image.qcow2', fmt=qcow2 size=4294967296 encryption=off > cluster_size=65536 lazy_refcounts=off refcount_bits=16 > > # ./qemu-io -c "write -P 0x22 0 1G" image.qcow2 > wrote 1073741824/1073741824 bytes at offset 0 > 1 GiB, 1 ops; 0:00:04.59 (222.886 MiB/sec and 0.2177 ops/sec) > > # ./qemu-img resize image.qcow2 512M > warning: qemu-img: Shrinking an image will delete all data beyond the > shrunken image's end. Before performing such an operation, make sure there is > no important data there. > error: qemu-img: Use the --shrink option to perform a shrink operation. > > # ./qemu-img resize --shrink image.qcow2 128M > Image resized. > > # ./qemu-img info image.qcow2 > image: image.qcow2 > file format: qcow2 > virtual size: 128M (134217728 bytes) > disk size: 128M > cluster_size: 65536 > Format specific information: > compat: 1.1 > lazy refcounts: false > refcount bits: 16 > corrupt: false > > # du -h image.qcow2 > 129Mimage.qcow2 > It looks sane to me, and already has a full set of R-Bs from Max. Are we waiting for Kevin? It looks like in Vladimir's series we sidestepped the problem of what happens if we resize with persistent bitmaps: we deny the operation, regardless of if we are shrinking, growing, or have bitmaps that are read-only, frozen, or what-have-you. It shouldn't be too hard to add soon, but this is fine for now. (I think for adding it we just need to make sure the bitmaps aren't frozen and are not read-only, and the implementation bitmap structure can already handle truncation in either direction just fine.) Anyway; Reviewed-by: John Snow > Changes from v1: > - add --shrink flag for qemu-img resize > - add qcow2_cache_discard > - simplify qcow2_shrink_l1_table() to reduce the likelihood of image > corruption > - add new qemu-iotests for shrinking images > > Changes from v2: > - replace qprintf() on error_report() (1) > - rewrite warning messages (1) > - enforce --shrink flag for all formats except raw (1) > - split qcow2_cache_discard() (2) > - minor fixes according to comments (3) > - rewrite the last part of qcow2_shrink_reftable() to avoid > qcow2_free_clusters() calls inside (3) > - improve test for shrinking image (4) > > Changes from v3: > - rebase on "Implement a warning_report function" Alistair's patch-set (1) > - spelling fixes (1) > - the man page fix according to the discussion (1) > - add call qcow2_signal_corruption() in case of image corruption (3) > > Changes from v4: > - rebase on https://github.com/XanClic/qemu/commits/block Max's block branch > > Changes from v5: > - the condition refcount == 0 should be enough to evict the l2/refcount > cluster > from the cache (2) > - overwrite the l1/refcount table in memory with zeros, even if overwriting > the > l1/refcount table on disk has failed (3) > - replace g_try_malloc() on g_malloc() for allocation reftable_tmp (3) > > Changes from v6: > - rebase on master 1f29673387 > > Pavel Butsykin (4): > qemu-img: add --shrink flag for resize > qcow2: add qcow2_cache_discard > qcow2: add shrink image support > qemu-iotests: add shrinking image test > > block/qcow2-cache.c| 26 +++ > block/qcow2-cluster.c | 50 + > block/qcow2-refcount.c | 140 - > block/qcow2.c | 43 +--- > block/qcow2.h | 17 + > qapi/block-core.json | 3 +- > qemu-img-cmds.hx | 4 +- > qemu-img.c | 23 ++ > qemu-img.texi | 6 +- > tests/qemu-iotests/102 | 4 +- > tests/qemu-iotests/163 | 170 > + > tests/qemu-iotests/163.out | 5 ++ > tests/qemu-iotests/group | 1 + > 13 files changed, 475 insertions(+), 17 deletions(-) > create mode 100644 tests/qemu-iotests/163 > create mode 100644 tests/qemu-iotests/163.out >
[Qemu-devel] [PATCH v7 0/4] Add shrink image for qcow2
This patch add shrinking of the image file for qcow2. As a result, this allows us to reduce the virtual image size and free up space on the disk without copying the image. Image can be fragmented and shrink is done by punching holes in the image file. # ./qemu-img create -f qcow2 image.qcow2 4G Formatting 'image.qcow2', fmt=qcow2 size=4294967296 encryption=off cluster_size=65536 lazy_refcounts=off refcount_bits=16 # ./qemu-io -c "write -P 0x22 0 1G" image.qcow2 wrote 1073741824/1073741824 bytes at offset 0 1 GiB, 1 ops; 0:00:04.59 (222.886 MiB/sec and 0.2177 ops/sec) # ./qemu-img resize image.qcow2 512M warning: qemu-img: Shrinking an image will delete all data beyond the shrunken image's end. Before performing such an operation, make sure there is no important data there. error: qemu-img: Use the --shrink option to perform a shrink operation. # ./qemu-img resize --shrink image.qcow2 128M Image resized. # ./qemu-img info image.qcow2 image: image.qcow2 file format: qcow2 virtual size: 128M (134217728 bytes) disk size: 128M cluster_size: 65536 Format specific information: compat: 1.1 lazy refcounts: false refcount bits: 16 corrupt: false # du -h image.qcow2 129Mimage.qcow2 Changes from v1: - add --shrink flag for qemu-img resize - add qcow2_cache_discard - simplify qcow2_shrink_l1_table() to reduce the likelihood of image corruption - add new qemu-iotests for shrinking images Changes from v2: - replace qprintf() on error_report() (1) - rewrite warning messages (1) - enforce --shrink flag for all formats except raw (1) - split qcow2_cache_discard() (2) - minor fixes according to comments (3) - rewrite the last part of qcow2_shrink_reftable() to avoid qcow2_free_clusters() calls inside (3) - improve test for shrinking image (4) Changes from v3: - rebase on "Implement a warning_report function" Alistair's patch-set (1) - spelling fixes (1) - the man page fix according to the discussion (1) - add call qcow2_signal_corruption() in case of image corruption (3) Changes from v4: - rebase on https://github.com/XanClic/qemu/commits/block Max's block branch Changes from v5: - the condition refcount == 0 should be enough to evict the l2/refcount cluster from the cache (2) - overwrite the l1/refcount table in memory with zeros, even if overwriting the l1/refcount table on disk has failed (3) - replace g_try_malloc() on g_malloc() for allocation reftable_tmp (3) Changes from v6: - rebase on master 1f29673387 Pavel Butsykin (4): qemu-img: add --shrink flag for resize qcow2: add qcow2_cache_discard qcow2: add shrink image support qemu-iotests: add shrinking image test block/qcow2-cache.c| 26 +++ block/qcow2-cluster.c | 50 + block/qcow2-refcount.c | 140 - block/qcow2.c | 43 +--- block/qcow2.h | 17 + qapi/block-core.json | 3 +- qemu-img-cmds.hx | 4 +- qemu-img.c | 23 ++ qemu-img.texi | 6 +- tests/qemu-iotests/102 | 4 +- tests/qemu-iotests/163 | 170 + tests/qemu-iotests/163.out | 5 ++ tests/qemu-iotests/group | 1 + 13 files changed, 475 insertions(+), 17 deletions(-) create mode 100644 tests/qemu-iotests/163 create mode 100644 tests/qemu-iotests/163.out -- 2.14.1