Always having ephemeral disks as raw might make sense given that there is going to be little overlap with others (and therefore the qcow2 benefits are not significant)
I guess this could also be a simple nova.conf flag per hypervisor with low risk (a default ephemeral format to qcow2 or raw). The l2 cache size option would still interesting for images. Tim On 23/10/15 19:36, "Marc Heckmann" <marc.heckm...@ubisoft.com> wrote: >Hi Daniel, > >On Fri, 2015-10-23 at 12:05 +0100, Daniel P. Berrange wrote: >> On Thu, Oct 22, 2015 at 06:01:42PM +0000, Marc Heckmann wrote: >> > Hi, >> > >> > On Thu, 2015-10-22 at 08:17 -0700, Abel Lopez wrote: >> > > I've actually looked for this for our RBD backed ephemeral instances, >> > > but found the options lacking. I last looked in Juno. >> > > >> > > On Thursday, October 22, 2015, Tim Bell <tim.b...@cern.ch> wrote: >> > > >> > > >> > > Has anyone had experience with setting up Nova with KVM so it >> > > has raw ephemeral disks but qcow2 images for the VMs ? We’ve >> > > got very large ephemeral disks and could benefit from the >> > > performance of raw volumes for this. >> > >> > We looked into this for the very same reasons and it doesn't seem to be >> > supported. >> > >> > That being said, I'm fearful of the boot time performance impact of >> > using RAW for ephemeral. >> >> There should be no performance impact of using fully pre-allocated >> raw images. Any decent modern filesystem (ext4, xfs) supports fallocate >> which allows you to pre-allocate an arbitrary sizes plain file in >> constant time. > >I was actually more afraid of the time it would take mkfs.<ext4|ntfs> to >create the filesystem. I was thinking that it might be non-negligible on >larger ephemeral sizes (approaching 1TB) going up to a minute or more. > >I just tested this and it doesn't seem to be an issue at all (obviously >--fast is used for NTFS). > >This would seem to make a setting to use raw only for ephemeral slightly >more attractive. > >That being said, if Nova gets fixed to pre-initialize qcow2 internally, >then the raison-d'être for raw might become less important. I'm assuming >the Nova fix for qcow2 would be easier to implement? Any downsides to >pre-initialized qcow2? > >In any case, thanks for the useful info! > >> >> By default Nova does *not* preallocate images - so both raw & qcow2 >> will grow-on-demand as guest writes to sectors. >> >> If the "preallocate_images" nova.conf option is set to "space", then >> Nova will call fallocate for both raw & qcow2 images to fully allocate >> the maximum space they require. There's no appreciable time overhead >> for this - it just prevents you overcommitting disk obviously. >> >> If you fully preallocate a qcow2 image its performance should pretty >> much match raw images (modulo the l2-cache-size item mentioned >> below), unfortunately, the way Nova is preallocating qcow2 images is >> wrong - it preallocates the space on disk, but does not pre-initialize >> the internal qcow2 data structures to match :-( So we need to fix >> that for qcow2 in Nova. >> >> > I suggest you check out the following presentation about qcow2 >> > performance if you haven't already done so: >> > >> > http://events.linuxfoundation.org/sites/events/files/slides/p0.pp_.pdf >> > >> > I think it would be worthwhile for Openstack (and libvirt if required) >> > to support the "l2-cache-size" option for qcow2. >> >> Yep, we should look at supporting that. >> >> >> Regards, >> Daniel > > >_______________________________________________ >OpenStack-operators mailing list >OpenStack-operators@lists.openstack.org >http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators _______________________________________________ OpenStack-operators mailing list OpenStack-operators@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators
