Oh yes, and storage overprovisioning doesn't currently work for KVM storage types, but it's a relatively simple fix:
https://issues.apache.org/jira/browse/CLOUDSTACK-5806 On Sat, Feb 1, 2014 at 12:12 AM, Marcus <[email protected]> wrote: > I don't think you'd have to change anything. Cloudstack (at least for > accounting purposes) considers all storage 'fat' and calculates > capacity per the disk offering size compared to total size of the > primary pool. Then you can add an overprovisioning factor to fit your > needs in the config options. > > On Sat, Feb 1, 2014 at 12:01 AM, Yoshikazu Nojima <[email protected]> wrote: >> Yes, preallocation=metadata creates a large file full of holes. and >> maybe I need to modify storage consumption measuring method of >> CloudStack to care this kind sparse file. >> >> To satisfy my needs, just turning on preallocation=metadata by default >> or making it configurable from agent.properties is enough, but I'm >> still thinking about making it configurable from disk offerings. >> >> Thanks, >> >> 2014-01-31 Marcus <[email protected]>: >>> Ok, so not technically sparse in the sense that you have a large file >>> full of holes, but it's still not allocating the entire disk upfront. >>> You get the same result of disk savings either way, existing >>> cloudstack installs with qcow2 are still saving space in the same >>> manner. >>> >>> On Fri, Jan 31, 2014 at 11:43 PM, Marcus <[email protected]> wrote: >>>> Here are my tests, using stock centos 6.5 qemu-img: >>>> >>>> [root@server ~]# qemu-img create -f qcow2 image.qcow2 10G >>>> Formatting 'image.qcow2', fmt=qcow2 size=10737418240 encryption=off >>>> cluster_size=65536 >>>> [root@server ~]# qemu-img info image.qcow2 >>>> image: image.qcow2 >>>> file format: qcow2 >>>> virtual size: 10G (10737418240 bytes) >>>> disk size: 136K >>>> cluster_size: 65536 >>>> >>>> No options leaves the disk as 136K, it's sparse. >>>> >>>> [root@server ~]# qemu-img create -f qcow2 -o preallocation=metadata >>>> imagepre.qcow2 10G >>>> Formatting 'imagepre.qcow2', fmt=qcow2 size=10737418240 encryption=off >>>> cluster_size=65536 preallocation='metadata' >>>> [root@server ~]# qemu info imagepre.qcow2 >>>> -bash: qemu: command not found >>>> [root@server ~]# qemu-img info imagepre.qcow2 >>>> image: imagepre.qcow2 >>>> file format: qcow2 >>>> virtual size: 10G (10737418240 bytes) >>>> disk size: 1.7M >>>> cluster_size: 65536 >>>> >>>> preallocation=metadata leaves disk as 1.7M, it's also sparse, but a bit >>>> bigger. >>>> >>>> On Fri, Jan 31, 2014 at 11:41 PM, Marcus <[email protected]> wrote: >>>>> Yes, no_option.img is only 256k in size. it's sparse. >>>>> >>>>> On Fri, Jan 31, 2014 at 11:38 PM, Yoshikazu Nojima <[email protected]> >>>>> wrote: >>>>>>>I don't think preallocation=metadata is required for sparse qcow2. >>>>>>>Just if you want it to NOT be sparse for metadata. >>>>>> >>>>>> I think it is required. Please see the output below. Only >>>>>> "metadata.img" is a sparse file. >>>>>> >>>>>> ============== >>>>>> [root@ut1-qmb-rd5c test]# qemu-img create -f qcow2 -o >>>>>> preallocation=off off.img 3G >>>>>> Formatting 'off.img', fmt=qcow2 size=3221225472 encryption=off >>>>>> cluster_size=65536 preallocation='off' >>>>>> [root@ut1-qmb-rd5c test]# qemu-img create -f qcow2 -o >>>>>> preallocation=metadata metadata.img 3G >>>>>> Formatting 'metadata.img', fmt=qcow2 size=3221225472 encryption=off >>>>>> cluster_size=65536 preallocation='metadata' >>>>>> [root@ut1-qmb-rd5c test]# qemu-img create -f qcow2 -o >>>>>> preallocation=full full.img 3G >>>>>> Formatting 'full.img', fmt=qcow2 size=3221225472 encryption=off >>>>>> cluster_size=65536 preallocation='full' >>>>>> [root@ut1-qmb-rd5c test]# qemu-img create -f qcow2 no_option.img 3G >>>>>> Formatting 'no_option.img', fmt=qcow2 size=3221225472 encryption=off >>>>>> cluster_size=65536 >>>>>> [root@ut1-qmb-rd5c test]# ls -alhs >>>>>> total 3.1G >>>>>> 4.0K drwxr-xr-x 2 root root 4.0K Feb 1 06:29 . >>>>>> 4.0K dr-xr-x---. 5 root root 4.0K Feb 1 06:19 .. >>>>>> 3.1G -rw-r--r-- 1 root root 3.1G Feb 1 06:23 full.img >>>>>> 592K -rw-r--r-- 1 root root 3.1G Feb 1 06:21 metadata.img >>>>>> 136K -rw-r--r-- 1 root root 256K Feb 1 06:29 no_option.img >>>>>> 136K -rw-r--r-- 1 root root 256K Feb 1 06:21 off.img >>>>>> ============== >>>>>> >>>>>> >>>>>>>I also remember hearing that either recent qemu-img has a decent >>>>>>>preallocation by default now, or that performance has improved such >>>>>>>that it doesn't matter. Will need to do some reading to remember, but >>>>>>>I think I remember hearing it's not a big deal nowadays. >>>>>> >>>>>> Ummm. I tested with "qemu-img version 0.12.1," and I confirmed the >>>>>> performance of "preallocation=off" is worse than >>>>>> "preallocation=metadata", >>>>>> but it may be not true with latest qemu. I will test it. >>>>>> >>>>>> Thank you for your comment! >>>>>> >>>>>> Regards, >>>>>> >>>>>> >>>>>> 2014-01-31 Marcus <[email protected]>: >>>>>>> I don't think preallocation=metadata is required for sparse qcow2. >>>>>>> Just if you want it to NOT be sparse for metadata. >>>>>>> >>>>>>> I also remember hearing that either recent qemu-img has a decent >>>>>>> preallocation by default now, or that performance has improved such >>>>>>> that it doesn't matter. Will need to do some reading to remember, but >>>>>>> I think I remember hearing it's not a big deal nowadays. >>>>>>> >>>>>>> On Fri, Jan 31, 2014 at 10:04 PM, Yoshikazu Nojima <[email protected]> >>>>>>> wrote: >>>>>>>> Hello Nux, >>>>>>>> >>>>>>>> Thank you for your comment. I will prepare feature specification. >>>>>>>> >>>>>>>> Regards, >>>>>>>> >>>>>>>> 2014-01-31 Nux! <[email protected]>: >>>>>>>>> On 31.01.2014 20:24, Yoshikazu Nojima wrote: >>>>>>>>>> >>>>>>>>>> Afternoon All, >>>>>>>>>> >>>>>>>>>> Is there anyone working on adding volume provisioning method option? >>>>>>>>>> As you know, thin provisioning of a volume save consumption of a >>>>>>>>>> storage, and fat provisioning improves IOPS performance. >>>>>>>>>> Especially, Qcow2 can save storage consumption and achive relatively >>>>>>>>>> better performance than default by provisioning a volume with an >>>>>>>>>> option "preallocation=metadata", which makes an image file a sparse >>>>>>>>>> file. >>>>>>>>>> >>>>>>>>>> http://itscblog.tamu.edu/improve-disk-io-performance-in-kvm/#.232---preallocation >>>>>>>>>> >>>>>>>>>> Any thoughts about this? >>>>>>>>>> If it is ok, I will write a feature specification on confluence and >>>>>>>>>> start implementation. >>>>>>>>>> >>>>>>>>>> Regards, >>>>>>>>>> Noji >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> Yoshikazu Nojima <[email protected]> >>>>>>>>> >>>>>>>>> >>>>>>>>> Hello, >>>>>>>>> >>>>>>>>> I thought preallocation=metadata is common practice since years. Now >>>>>>>>> I find >>>>>>>>> out ACS doesn't actually use it? >>>>>>>>> If so, this is really _bad_ and needs to be fixed ASAP... >>>>>>>>> >>>>>>>>> Thanks Noji >>>>>>>>> >>>>>>>>> -- >>>>>>>>> Sent from the Delta quadrant using Borg technology! >>>>>>>>> >>>>>>>>> Nux! >>>>>>>>> www.nux.ro
