When many virtio-blk devices are present, managed IRQ reservations can exhaust the per-CPU interrupt vector space and cause probe to fail with -ENOSPC:
virtio_blk virtioNNN: probe with driver virtio_blk failed with error -28 On x86, the affinity masks of the managed request-queue interrupts collectively cover the possible CPUs. irq_matrix_reserve_managed() therefore reserves one vector slot per possible CPU for each device, even when the device has far fewer queues than CPUs. Once a CPU's vector space is exhausted, further managed reservations fail. Add use_irq_affinity, defaulting to true to keep existing behavior. Setting it to false makes the request-queue interrupts unmanaged, so each interrupt consumes a vector on only one CPU. The vector allocator places each unmanaged interrupt on the least-loaded CPU, allowing the interrupts to be spread across CPUs. For example, the reported 64-CPU system used one request queue per device, so each device needs two interrupts (config-change and request queue). With managed affinity disabled, 800 devices consume about 800 * 2 / 64 = 25 vector slots per CPU. This stays well below the per-CPU vector limit and allows more devices to probe. Signed-off-by: Liu, Changcheng <[email protected]> Reviewed-by: Stefan Hajnoczi <[email protected]> diff --git a/drivers/block/virtio_blk.c b/drivers/block/virtio_blk.c index 32bf3ba07a9d..184f1c1f4485 100644 --- a/drivers/block/virtio_blk.c +++ b/drivers/block/virtio_blk.c @@ -41,6 +41,11 @@ static unsigned int poll_queues; module_param(poll_queues, uint, 0644); MODULE_PARM_DESC(poll_queues, "The number of dedicated virtqueues for polling I/O"); +static bool use_irq_affinity = true; +module_param(use_irq_affinity, bool, 0644); +MODULE_PARM_DESC(use_irq_affinity, + "Use managed IRQ affinity for virtqueues (default: true)"); + static int major; static DEFINE_IDA(vd_index_ida); @@ -1016,7 +1021,8 @@ static int init_vq(struct virtio_blk *vblk) } /* Discover virtqueues and write information to configuration. */ - err = virtio_find_vqs(vdev, num_vqs, vqs, vqs_info, &desc); + err = virtio_find_vqs(vdev, num_vqs, vqs, vqs_info, + use_irq_affinity ? &desc : NULL); if (err) goto out; -- 2.43.7
