Applying I/O limits to a disk that is already serving requests from an
iothread can kill QEMU:
#0 throttle_group_next_tgm (tgm=0x0) at block/throttle-groups.c:185
#1 next_throttle_token (tgm=..., direction=THROTTLE_READ)
#2 throttle_group_co_io_limits_intercept (bytes=8192, ...)
#3 blk_co_do_preadv_part (blk=..., bytes=8192, ...)
#4 blk_aio_read_entry (opaque=...)
#5 coroutine_trampoline (i0=..., i1=...)
throttle_group_register_tgm() publishes tgm->throttle_state before it
takes tg->lock, and the group it points at is only filled in afterwards.
A request that samples the pointer in between and wins tg->lock finds
tg->tokens[] still NULL. The ordering is as old as the throttle group
code, 76f4afb40f ("throttle: Add throttle group support").
This was found from a production core. The group in it was brand new -
name matching the disk, refcount 1, empty member list, both tokens NULL,
a zeroed config because throttle_group_config() had not run yet, and
tg->lock owned by the crashing iothread while the main thread sat
blocked on the same lock inside throttle_group_register_tgm(). The
trigger was libvirt applying blkdeviotune right after attaching a
volume, while the guest was probing the new disk.
The window is a couple of instructions wide. To see the bug on demand,
widen it by hand:
--- a/block/throttle-groups.c
+++ b/block/throttle-groups.c
@@ -584,6 +584,7 @@ void throttle_group_register_tgm(...)
tgm->throttle_state = ts;
tgm->aio_context = ctx;
qatomic_set(&tgm->restart_pending, 0);
+ g_usleep(1000);
QEMU_LOCK_GUARD(&tg->lock);
With that in place, any I/O against the disk during a
block_set_io_throttle that creates the group crashes at once, and the
patch below makes it stop.
Cc: Kevin Wolf <[email protected]>
Cc: Hanna Reitz <[email protected]>
Cc: Alberto Garcia <[email protected]>
Signed-off-by: Denis V. Lunev <[email protected]>
Denis V. Lunev (1):
block/throttle-groups: fix crash when enabling I/O limits on a busy
disk
block/block-backend.c | 12 ++++++------
block/throttle-groups.c | 4 +++-
2 files changed, 9 insertions(+), 7 deletions(-)
--
2.53.0
