On Wed, 15 Nov 2023 at 11:57, Mike Christie <michael.chris...@oracle.com> wrote: > > On 11/15/23 6:57 AM, Stefan Hajnoczi wrote: > > On Wed, Nov 15, 2023 at 12:43:02PM +0100, Stefano Garzarella wrote: > >> On Mon, Nov 13, 2023 at 06:36:44PM -0600, Mike Christie wrote: > >>> This adds support for vhost-scsi to be able to create a worker thread > >>> per virtqueue. Right now for vhost-net we get a worker thread per > >>> tx/rx virtqueue pair which scales nicely as we add more virtqueues and > >>> CPUs, but for scsi we get the single worker thread that's shared by all > >>> virtqueues. When trying to send IO to more than 2 virtqueues the single > >>> thread becomes a bottlneck. > >>> > >>> This patch adds a new setting, virtqueue_workers, which can be set to: > >>> > >>> 1: Existing behavior whre we get the single thread. > >>> -1: Create a worker per IO virtqueue. > >> > >> I find this setting a bit odd. What about a boolean instead? > >> > >> `per_virtqueue_workers`: > >> false: Existing behavior whre we get the single thread. > >> true: Create a worker per IO virtqueue. > > > > Me too, I thought there would be round-robin assignment for 1 < > > worker_cnt < (dev->nvqs - VHOST_SCSI_VQ_NUM_FIXED) but instead only 1 > > and -1 have any meaning. > > > > Do you want to implement round-robin assignment? > > > > It was an int because I originally did round robin but at some point > dropped it. I found that our users at least: > > 1. Are used to configuring number of virtqueues. > 2. In the userspace guest OS are used to checking the queue to CPU > mappings to figure out how their app should optimize itself. > > So users would just do a virtqueue per vCPU or if trying to reduce > mem usage would do N virtqueues < vCPUs. For both cases they just did the > worker per virtqueue. > > However, I left it an int in case in the future someone wanted > the future.
Okay. I have a slight preference for a vq_workers bool parameter in that case. An int parameter can be added later if the number of workers need to be specified. But if you want to keep -1/1 then I guess it's fine unless someone else has a strong opinion. Stefan
