Hi all!
Here is a filter, which does preallocation on write.
v7:
01: add Alberto's r-b
07: don't remove sentence from the comment
08: - drop extra "s->file_end = end;" line
- improve check/set perm handlers
09: add Max's r-b
10: add Max's r-b
11: new
12: - skip if preallocate unsupported
- drop auto and quick groups
13: new
14: - improve 'average' field of result spec
- drop extra 'dim = ...' line
15-18: new
19: a lot of changes
20: new
21: add results dump to json
In Virtuozzo we have to deal with some custom distributed storage
solution, where allocation is very-very expensive operation. We have to
workaround it in Qemu, so here is a new filter.
Still, the filter shows good results for me even for xfs and ext4.
Here are results, produced by new benchmark (last several patches):
All results are in iops (larger means better)
---------------------------------- ------------- ------------
no-prealloc prealloc
ssd-ext4, aligned sequential 16k 2e+04 2.7e+04
+36%
hdd-ext4, aligned sequential 16k 4.8e+03 ± 5% 3.1e+03 ± 4%
-36%
ssd-xfs, aligned sequential 16k 1.6e+04 ± 5% 2.5e+04 ± 3%
+59%
hdd-xfs, aligned sequential 16k 4.7e+03 ± 22% 3.2e+03 ± 3%
-31%
ssd-ext4, unaligned sequential 64k 2e+04 2.7e+04
+36%
hdd-ext4, unaligned sequential 64k 4.9e+03 ± 2% 3.2e+03 ± 8%
-34%
ssd-xfs, unaligned sequential 64k 1.5e+04 2.6e+04 ± 2%
+69%
hdd-xfs, unaligned sequential 64k 5.2e+03 ± 2% 3.3e+03 ± 5%
-35%
---------------------------------- ------------- ------------
Note: it's on Fedora 30, kernel 5.6.13-100.fc30.x86_64
The tests are actually qemu-img bench, run like:
./qemu-img create -f qcow2 $img 16G
aligned:
./qemu-img bench -c 10000 -d 64 -f qcow2 -s 16k -t none -n -w $img
unaligned
./qemu-img bench -c 10000 -d 64 -f qcow2 -o 1k -s 64k -t none -n -w $img
and for preallocation, you'll drop -f qcow2, add --image-opts, and
instead of just $img use
driver=qcow2,file.driver=preallocate,file.file.driver=file,file.file.filename=$img
Or, use new benchmark like this:
./bench_prealloc.py ../../build/qemu-img \
ssd-ext4:/path/to/mount/point \
ssd-xfs:/path2 hdd-ext4:/path3 hdd-xfs:/path4
Vladimir Sementsov-Ogievskiy (21):
block: simplify comment to BDRV_REQ_SERIALISING
block/io.c: drop assertion on double waiting for request serialisation
block/io: split out bdrv_find_conflicting_request
block/io: bdrv_wait_serialising_requests_locked: drop extra bs arg
block: bdrv_mark_request_serialising: split non-waiting function
block: introduce BDRV_REQ_NO_WAIT flag
block: bdrv_check_perm(): process children anyway
block: introduce preallocate filter
qemu-io: add preallocate mode parameter for truncate command
iotests: qemu_io_silent: support --image-opts
iotests.py: execute_setup_common(): add required_fmts argument
iotests: add 298 to test new preallocate filter driver
scripts/simplebench: fix grammar: s/successed/succeeded/
scripts/simplebench: support iops
scripts/simplebench: use standard deviation for +- error
simplebench: rename ascii() to results_to_text()
simplebench: move results_to_text() into separate file
simplebench/results_to_text: improve view of the table
simplebench/results_to_text: add difference line to the table
simplebench/results_to_text: make executable
scripts/simplebench: add bench_prealloc.py
docs/system/qemu-block-drivers.rst.inc | 26 ++
qapi/block-core.json | 20 +-
include/block/block.h | 20 +-
include/block/block_int.h | 3 +-
block.c | 7 +-
block/file-posix.c | 2 +-
block/io.c | 130 +++---
block/preallocate.c | 559 +++++++++++++++++++++++++
qemu-io-cmds.c | 46 +-
block/meson.build | 1 +
scripts/simplebench/bench-example.py | 3 +-
scripts/simplebench/bench_prealloc.py | 132 ++++++
scripts/simplebench/bench_write_req.py | 3 +-
scripts/simplebench/results_to_text.py | 126 ++++++
scripts/simplebench/simplebench.py | 66 ++-
tests/qemu-iotests/298 | 186 ++++++++
tests/qemu-iotests/298.out | 5 +
tests/qemu-iotests/group | 1 +
tests/qemu-iotests/iotests.py | 16 +-
19 files changed, 1225 insertions(+), 127 deletions(-)
create mode 100644 block/preallocate.c
create mode 100755 scripts/simplebench/bench_prealloc.py
create mode 100755 scripts/simplebench/results_to_text.py
create mode 100644 tests/qemu-iotests/298
create mode 100644 tests/qemu-iotests/298.out
--
2.21.3
