On Thu, 01/22 14:14, Max Reitz wrote: > On 2015-01-19 at 08:16, Jun Li wrote: > >On Thu, 01/15 13:47, Max Reitz wrote: > >>On 2015-01-03 at 07:23, Jun Li wrote: > >>>On Fri, 11/21 11:56, Max Reitz wrote: > >>>>So, as for what I think we do need to do when shrinking (and keep in mind: > >>>>The offset given to qcow2_truncate() is the guest size! NOT the host image > >>>>size!): > >>>> > >>>>(1) Determine the first L2 table and the first entry in the table which > >>>>will > >>>>lie beyond the new guest disk size. > >>>Here is not correct always. Due to the COW, using offset to calculate the > >>>first entry of the first L2 table will be incorrect. > >>Again: This is *not* about the host disk size or the host offset of some > >>cluster, but about the *guest* disk size. > >> > >>Let's make up an example. You have a 2 GB disk but you want to resize it to > >>1.25 GB. The cluster size is 64 kB, therefore we have 2 GB / 64 kB = 32,768 > >>data clusters (as long as there aren't any internal snapshots, which is a > >>prerequisite for resizing qcow2 images). > >> > >>Every L2 table contains 65,536 / 8 = 8,192 entries; there are thus 32,768 / > >>8,192 = 4 L2 tables. > >> > >>As you can see, one can directly derive the number of data clusters and L2 > >>tables from the guest disk size (as long as there aren't any internal > >>snapshots). > >> > >>So of course we can do the same for the target disk size: 1.25 GB / 64 kB = > >>20,480 data clusters; 20,480 / 8,192 = 2.5 L2 tables, therefore we need > >>three L2 tables but only half of the last one (4,096 entries). > >> > >Sorry, last time is my mis-understanding. If do not use qcow2_truncate(), I > >think don't existing above issue. > > > >For my original thought, I want to say: > >Sometimes the second L2 table will contain some entry, the pointer in this > >entry will point to a cluster which address is larger than 1.25 GB. > > Correct. > > >So if not use qcow2_truncate(), won't discard above cluster which address is > >larger than 1.25 GB.
Sorry, I do not express my meaning clearly. Here I want to say: As some entry(let call it entry1) will point to a cluster(let call it cluster1) which address is larger than 1.25GB, so if we use qcow2_truncate() and will discard this cluster1. So entry1 will have an error after cluster1 discard. If do not use qcow2_truncate(), so won't discard cluster1. > > I'm sorry, I can't really follow what you are trying to say here, so I'll > just try to reply with things that may or may not be what you wanted to talk > about. > > If you are using qemu-img resize and thus subsequently qcow2_truncate() to > shrink an image, you cannot expect the image to shrink to the specified file > length, for several reasons. > > First, if you shrink it to 1 GB, but only half of that is actually used, the > image might of course very well have a length below 1 GB. > > Second, there is metadata overhead. So if you are changing the guest disk > size to 1 GB (all of which is occupied), the host file size will exceed 1 GB > because of that overhead. > > Third, I keep repeating myself here, but file length is not file size. So > you may observe a file length of 10 GB or more because the clusters are > spread all over the image file. This is something we'd have to combat with > defragmentation; but the question is whether we really need to (see below > for more on that). The point is that it doesn't matter whether the image has > a file length of 10 GB; the file size will be around 1 GB anyway. > > >But I still have another worry. > > > >Suppose "virtual size" and "disk size" are all 2G. After we resize it to > >1.25G, seems we will get "virtual size" is 1.25G but "disk size" is still 2G > > No, it won't. I can prove it to you: Yes, you are right. I have double checked my PATCH v5. Seems I don't use qcow2_process_discards(and this function will call bdrv_discard() to discard cluster on host) in my patch v5. I will submit a new version of patch. Thanks. Regards, Jun Li > > $ qemu-img create -f qcow2 test.qcow2 64M > $ qemu-io -c 'write 0 64M' test.qcow2 > $ qemu-img info test.qcow2 > ... > disk size: 64M > ... > > Okay, so far it's just what we'd expect. Now let's implement my proposal for > truncation: Let's assume the image should be shrinked to 32 MB, so we > discard all clusters starting at 32 MB (guest offset) (which is 64 MB - 32 > MB = 32 MB of data): > > $ qemu-io -c 'discard 32M 32M' test.qcow2 > $ qemu-img info test.qcow2 > ... > disk size: 32M > ... > > Great! > > >if do not use "qcow2_truncate()" to truncate the file(Yes, I know use > >qcow2_truncate is not a resolution). This seems strange, not so perfect. > > > >>We know that every cluster references somewhere after that limit (that is, > >>every entry in the fourth L2 table and every entry starting with index 4,096 > >>in the third L2 table) is a data cluster with a guest offset somewhere > >>beyond 1.25 GB, so we don't need it anymore. > >> > >>Thus, we simply discard all those data clusters and after that we can > >>discard the fourth L2 table. That's it. > >> > >>If we really want to we can calculate the highest cluster host offset in use > >>and truncate the image accordingly. But that's optional, see the last point > >>in my "problems with this approach" list (having discarded the clusters > >>should save us all the space already). Furthermore, as I'm saying in that > >>list, to really solve this issue, we'd need qcow2 defragmentation. > >> > >Do we already have "qcow2 defragmentation" realization? > > No, we don't. The only way to defragment a qcow2 image right now is using > qemu-img convert to create a (defragmented) copy and then delete the old > image, which has the disadvantage of temporarily requiring double the disk > space and being an offline operation. > > So far, nobody has implemented online defragmentation, mainly for two > reasons: It would probably be pretty complicated (it'd probably need to be a > block job which links into a pretty low-level function provided by qcow2 > (defragment_some_clusters or something)) and second, so far there has been > little demand. Disk space is not an issue (as said before), because it > doesn't really matter to a modern file system whether your file has a length > of 100 MB of 100 GB; that's just some number. What really matters is how > much of that space is actually used; and if all unused clusters are > discarded, there won't be any space used for them (well, maybe there is some > metadata overhead, but that should be negligible). > > There are a couple of reasons why you'd want to defragment an image: > > First, it makes you feel better. I can relate to that, but it's not a real > reason. > > Second, it may improve performance: The guest may expect consecutive reads > to be fast; but if the clusters are sprinkled all over the host, consecutive > guest reads no longer necessarily translate to consecutive reads on the host > (same for writes, of course). Defragmentation would probably fix that, but > if you want to rely on this, you'd better use preallocated image files. > > Third, it looks better. People expect the file length to be raw indicator of > the file size. However, for me this is related to "it makes you feel > better", because this also is not a really good reason. > > Fourth, using a non-modern file system may let your file size explode > because suddenly, file length is actually equal to the file size. But I > think, in this case you should just use a better file system. > > I don't know whether "cp" copies holes in files; its manpage says it does > create sparse images, but I don't know how well it works; but I just assume > it works well enough. > > Max > > >Jun Li >
