Re: Can vhost translate to io_uring?
On 6/14/23 1:25 AM, michael.chris...@oracle.com wrote: > It would be nice if the vhost layer could use the io-wq code as sort of > generic worker. I can look into what that would take if Jens is ok > with that type of thing. We could use the io-wq code, but we hit the same problems as before: 1. We still need to modify the vhost drivers like I mentioned below so when the task gets SIGKILL the drivers fail instead of running the work like normal. 2. We still need some code like the patch below so when the worker task exits and is freed the vhost drivers stop calling io_wq_enqueue and don't access the io_wq anymore. 3. There's some other small things which seem easy to change like we need to create the worker thread/task_struct when io_wq_create is run instead of io_wq_enqueue. The problem is that we can queue work from threads that have different mms than we want to use. I've done #2 in the patch below. I'm almost done with #1. Just testing it now. When that's done, we can remove the signal hacks and then decide if we want to go further and switch to io-wq. > > For vhost, I just submitted a patch to the vhost layer that allow us to > switch out the vhost worker thread when IO is running: > > https://lists.linuxfoundation.org/pipermail/virtualization/2023-June/067246.html > > After that patch, I just need to modify vhost_worker/vhost_task_fn so > when get_signal returns true we set the worker to NULL and do a > synchronize_rcu. > Then I just need to modify vhost-scsi so it detects when the worker is NULL > and modify the flush code so it handles work that didn't get to run. > > > ___ Virtualization mailing list Virtualization@lists.linux-foundation.org https://lists.linuxfoundation.org/mailman/listinfo/virtualization
Re: Can vhost translate to io_uring?
On Wed, Jun 14, 2023 at 01:02:58AM -0500, Eric W. Biederman wrote: > At a quick glance it looks like io_uring already supports the > functionality that vhost supports (which I think is networking and > scsi). There's vsock too. -- MST ___ Virtualization mailing list Virtualization@lists.linux-foundation.org https://lists.linuxfoundation.org/mailman/listinfo/virtualization
Re: Can vhost translate to io_uring?
On 6/14/23 12:25?AM, michael.chris...@oracle.com wrote: > On 6/14/23 1:02 AM, Eric W. Biederman wrote: >> >> I am sad my idea for simplifying things did not work out. >> >> >> Let's try an even bigger idea to reduce maintenance and simplify things. >> >> Could vhost depend on io_uring? >> >> Could vhost just be a translation layer of existing vhost requests to >> io_uring requests? >> >> At a quick glance it looks like io_uring already supports the >> functionality that vhost supports (which I think is networking and >> scsi). >> >> If vhost could become a translation layer that would allow removing >> the vhost worker and PF_USER_WORKER could be removed completely, >> leaving only PF_IO_WORKER. >> >> >> I suggest this because a significant vhost change is needed because in > > It would be nice if the vhost layer could use the io-wq code as sort of > generic worker. I can look into what that would take if Jens is ok > with that type of thing. Certainly. io-wq is mostly generic, eg it has no understanding of io_uring internals or commands and structs, and it should be possible to just setup a struct io_wq and use that. Obviously might need a bit of refactoring work and exporting of symbols, io_uring is y/n so we don't export anything. But I think it should all be minor work, really. -- Jens Axboe ___ Virtualization mailing list Virtualization@lists.linux-foundation.org https://lists.linuxfoundation.org/mailman/listinfo/virtualization
Re: Can vhost translate to io_uring?
On Wed, Jun 14, 2023 at 01:02:58AM -0500, Eric W. Biederman wrote: > > I am sad my idea for simplifying things did not work out. > > > Let's try an even bigger idea to reduce maintenance and simplify things. > > Could vhost depend on io_uring? > > Could vhost just be a translation layer of existing vhost requests to > io_uring requests? I expect that's going to have a measureable performance impact. > At a quick glance it looks like io_uring already supports the > functionality that vhost supports (which I think is networking and > scsi). > > If vhost could become a translation layer that would allow removing > the vhost worker and PF_USER_WORKER could be removed completely, > leaving only PF_IO_WORKER. > > > I suggest this because a significant vhost change is needed because in > the long term the hacks in exec and coredump are not a good idea. Which > means something like my failed "[PATCH v3] fork, vhost: Use CLONE_THREAD > to fix freezer/ps regression". > > If we have to go to all of the trouble of reworking things it why can't > we just make io_uring do all of the work? > > Eric ___ Virtualization mailing list Virtualization@lists.linux-foundation.org https://lists.linuxfoundation.org/mailman/listinfo/virtualization
Re: Can vhost translate to io_uring?
On 6/14/23 1:02 AM, Eric W. Biederman wrote: > > I am sad my idea for simplifying things did not work out. > > > Let's try an even bigger idea to reduce maintenance and simplify things. > > Could vhost depend on io_uring? > > Could vhost just be a translation layer of existing vhost requests to > io_uring requests? > > At a quick glance it looks like io_uring already supports the > functionality that vhost supports (which I think is networking and > scsi). > > If vhost could become a translation layer that would allow removing > the vhost worker and PF_USER_WORKER could be removed completely, > leaving only PF_IO_WORKER. > > > I suggest this because a significant vhost change is needed because in It would be nice if the vhost layer could use the io-wq code as sort of generic worker. I can look into what that would take if Jens is ok with that type of thing. For vhost, I just submitted a patch to the vhost layer that allow us to switch out the vhost worker thread when IO is running: https://lists.linuxfoundation.org/pipermail/virtualization/2023-June/067246.html After that patch, I just need to modify vhost_worker/vhost_task_fn so when get_signal returns true we set the worker to NULL and do a synchronize_rcu. Then I just need to modify vhost-scsi so it detects when the worker is NULL and modify the flush code so it handles work that didn't get to run. > the long term the hacks in exec and coredump are not a good idea. Which > means something like my failed "[PATCH v3] fork, vhost: Use CLONE_THREAD > to fix freezer/ps regression". > > If we have to go to all of the trouble of reworking things it why can't > we just make io_uring do all of the work? > > Eric ___ Virtualization mailing list Virtualization@lists.linux-foundation.org https://lists.linuxfoundation.org/mailman/listinfo/virtualization
Can vhost translate to io_uring?
I am sad my idea for simplifying things did not work out. Let's try an even bigger idea to reduce maintenance and simplify things. Could vhost depend on io_uring? Could vhost just be a translation layer of existing vhost requests to io_uring requests? At a quick glance it looks like io_uring already supports the functionality that vhost supports (which I think is networking and scsi). If vhost could become a translation layer that would allow removing the vhost worker and PF_USER_WORKER could be removed completely, leaving only PF_IO_WORKER. I suggest this because a significant vhost change is needed because in the long term the hacks in exec and coredump are not a good idea. Which means something like my failed "[PATCH v3] fork, vhost: Use CLONE_THREAD to fix freezer/ps regression". If we have to go to all of the trouble of reworking things it why can't we just make io_uring do all of the work? Eric ___ Virtualization mailing list Virtualization@lists.linux-foundation.org https://lists.linuxfoundation.org/mailman/listinfo/virtualization
