2010-06-24 00:16:03 -, Jamie Lokier:
Serge Hallyn wrote:
The default of qemu-img (of using O_SYNC) is not very sensible
because anyway, the client (the kernel) uses caches (write-back),
(and qemu-nbd -d doesn't flush those by the way). So if for
instance qemu-nbd is killed, regardless of whether qemu-nbd uses
O_SYNC, O_DIRECT or not, the data in the image will not be
consistent anyway, unless syncs are done by the client (like fsync
on the nbd device or sync mount option), and with qemu-nbd's O_SYNC
mode, those syncs will be extremely slow.
Do the client syncs cause the nbd server to fsync or fdatasync the
file?
The clients syncs cause the data to be sent to the server. The
server then writes it to disk and each write blocks until the
data is written physically on disk with O_SYNC.
It appears it is because by default the disk image it serves is open
with O_SYNC. The --nocache option, unintuitively, makes matters a
bit better because it causes the image to be open with O_DIRECT
instead of O_SYNC.
[...]
--cache=off is the same as --nocache (that is use O_DIRECT),
writethrough is using O_SYNC and is still the default so this patch
doesn't change the functionality. writeback is none of those flags,
so is the addition of this patch. The patch also does an fsync upon
qemu-nbd -d to make sure data is flushed to the image before
removing the nbd.
I really wish qemu's options didn't give the false impression
nocache does less caching than writethrough. O_DIRECT does
caching in the disk controller/hardware, while O_SYNC hopefully does
not, nowadays.
[...]
Note that I use the same none, writethrough, writeback as
another utility shipped with qemu for consistency (see vl.c in
the source), I don't mind about the words as long as the
writeback functionality is available.
Cheers,
Stephane
--
qemu-nbd slow and missing writeback cache option
https://bugs.launchpad.net/bugs/595117
You received this bug notification because you are a member of qemu-
devel-ml, which is subscribed to QEMU.
Status in QEMU: Invalid
Status in “qemu-kvm” package in Ubuntu: Incomplete
Bug description:
Binary package hint: qemu-kvm
dpkg -l | grep qemu
ii kvm
1:84+dfsg-0ubuntu16+0.12.3+noroms+0ubuntu9dummy transitional
pacakge from kvm to qemu-
ii qemu 0.12.3+noroms-0ubuntu9
dummy transitional pacakge from qemu to qemu
ii qemu-common 0.12.3+noroms-0ubuntu9
qemu common functionality (bios, documentati
ii qemu-kvm 0.12.3+noroms-0ubuntu9
Full virtualization on i386 and amd64 hardwa
ii qemu-kvm-extras 0.12.3+noroms-0ubuntu9
fast processor emulator binaries for non-x86
ii qemu-launcher1.7.4-1ubuntu2
GTK+ front-end to QEMU computer emulator
ii qemuctl 0.2-2
controlling GUI for qemu
lucid amd64.
qemu-nbd is a lot slower when writing to disk than say nbd-server.
It appears it is because by default the disk image it serves is open with
O_SYNC. The --nocache option, unintuitively, makes matters a bit better because
it causes the image to be open with O_DIRECT instead of O_SYNC.
The qemu code allows an image to be open without any of those flags, but
unfortunately qemu-nbd doesn't have the option to do that (qemu doesn't allow
the image to be open with both O_SYNC and O_DIRECT though).
The default of qemu-img (of using O_SYNC) is not very sensible because anyway,
the client (the kernel) uses caches (write-back), (and qemu-nbd -d doesn't
flush those by the way). So if for instance qemu-nbd is killed, regardless of
whether qemu-nbd uses O_SYNC, O_DIRECT or not, the data in the image will not
be consistent anyway, unless syncs are done by the client (like fsync on the
nbd device or sync mount option), and with qemu-nbd's O_SYNC mode, those
syncs will be extremely slow.
Attached is a patch that adds a --cache={off,none,writethrough,writeback}
option to qemu-nbd.
--cache=off is the same as --nocache (that is use O_DIRECT), writethrough is
using O_SYNC and is still the default so this patch doesn't change the
functionality. writeback is none of those flags, so is the addition of this
patch. The patch also does an fsync upon qemu-nbd -d to make sure data is
flushed to the image before removing the nbd.
Consider this test scenario:
dd bs=1M count=100 of=a /dev/null
qemu-nbd --cache=x -c /dev/nbd0 a
cp /dev/zero /dev/nbd0
time perl -MIO::Handle -e 'STDOUT-sync or die$!' 1 /dev/nbd0
With cache=writethrough (the default), it takes over 10 minutes to write those
100MB worth of zeroes. Running a strace, we see the recvfrom and sentos delayed
by each 1kb write(2)s