Re: [Openstack-operators] Issue with snapshots of raw images

2017-03-14 Thread George Shuklin
If anyone curious about this bug: It was clearly linux-3.13  bug, issue 
was completely solved by moving to 4.4.


On 02/15/2017 08:03 PM, George Shuklin wrote:

Hello.

We've upgraded to mitaka, qemu 2.5/linux-3.13 and found that raw 
images now have BIG issues with snapshots.


Symptoms:

When snapshot process reach 'fallocated' blocks (see below) all IO in 
guest start lagging, including network IO. Windows starts loosing 
pings for very long period of time (~30-40 minutes), linux do this 
briefly (~500-700ms, but regularly).


Research so far:

All those symptoms may be resolved if every disk block is actually 
written on disk (dd if=disk of=disk conv=notrunc). If file has 
fallocated blocks, it will cause problem. If it has sparse hole it 
will cause problem too, but when preallocate_images = space, no any 
sparse holes are in the file.


Best way so far to distinguish 'bad' disk from 'good' is to use 
filefrag -v. For 'bad' disk it shows "unwritten" flag.


1. Any idea how to prevent this?

2. Any idea how to force nova to actually write images completely 
without using 'fallocate'?





___
OpenStack-operators mailing list
OpenStack-operators@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators


[Openstack-operators] Issue with snapshots of raw images

2017-02-15 Thread George Shuklin

Hello.

We've upgraded to mitaka, qemu 2.5/linux-3.13 and found that raw images 
now have BIG issues with snapshots.


Symptoms:

When snapshot process reach 'fallocated' blocks (see below) all IO in 
guest start lagging, including network IO. Windows starts loosing pings 
for very long period of time (~30-40 minutes), linux do this briefly 
(~500-700ms, but regularly).


Research so far:

All those symptoms may be resolved if every disk block is actually 
written on disk (dd if=disk of=disk conv=notrunc). If file has 
fallocated blocks, it will cause problem. If it has sparse hole it will 
cause problem too, but when preallocate_images = space, no any sparse 
holes are in the file.


Best way so far to distinguish 'bad' disk from 'good' is to use filefrag 
-v. For 'bad' disk it shows "unwritten" flag.


1. Any idea how to prevent this?

2. Any idea how to force nova to actually write images completely 
without using 'fallocate'?



___
OpenStack-operators mailing list
OpenStack-operators@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators