On Fri, 2 Dec 2016 13:58:08 -0500 Chris Metcalf <cmetc...@mellanox.com> wrote:
> On 12/1/2016 5:28 PM, yunhong jiang wrote: > > Hi, Chris > > I noticed your task isolation patch set at > > https://lkml.org/lkml/2016/8/9/759 . Thanks a lot for the great > > effort. > > > > I checked the patch and have some questions about how to use > > this functionality on the NFV environment. In the NFV scenario, > > high speed network code runs on a VM user space and the VM is > > hosted by KVM hypervisor. To achieve the high speed network, not > > only the network code should be isolated, also the vCPU thread. > > That's true. > > > I checked your patch to think how to isolate the vCPU thread, > > and I have some questions and hope your hints: > > > > a) If the task isolation need prctl to mark itself as isolated, > > possibly the vCPU thread can't achieve it. First, the vCPU thread > > may need system service during OS booting time, also it's the > > application, instead of the vCPU thread to decide if the vCPU > > thread should be isolated. So possibly we need a mechanism so that > > another process can set the vCPU thread's task isolation? > > These are good questions. I think that the we would probably want to > add a KVM mode that did the prctl() before transitioning back to the Would prctl() when back to gues be too heavy? > guest. But then, in the same way that we currently allow another > prctl() from a task-isolated userspace process, we'd probably need to You mean currently in your patch we alraedy can do the prctl from 3rd party process to task-isolate a userspace process? Sorry that I didn't notice that part. > allow a KVM exit from a guest to happen without triggering any > task-isolation errors. yes. > > Alternately, if we add the proposed NOSIG mode, then the hypervisor > can use that, and simply stay in task-isolation mode no matter what > system support is requested. Yes, possibly we will keep the vCPU thread as always NOSIG mode. > > Presumably in the guest, we'd run task isolation as well, but > presumably we'd only exit to the hypervisor if we were already in the > guest kernel? Although I guess you could image providing trapping > mmio mappings to the guest's userspace process that caused a > hypervisor exit. Perhaps we extend the hypervisor to know that a > guest might support task isolation, and to generate a warning on > behalf of the guest if the hypervisor sees an exit from the guest's > userspace and the guest task is in task-isolation mode? > > > b) I noticed that curently the task_isolation_prepare() is invoked > > on exit_to_usermode_loop(), while we may need such job on VM Exit > > procedure, or interrupt handling during VM exit handling. > > Yes, something like that. I am not super-familiar with the KVM > internals (I did a KVM port to the tile architecture a while back, but > I'd have to swap all that memory back in before I could even have a > half-educated opinion.) Got it. > > > Also, I'm also considering how to utilize this task_isolation > > for network node which is not busy-loop, like the interrupt mode > > DPDK code. In the interrupt mode DPDK code, the application may > > yield the CPU if there is no network packet for a very long time > > and it will re-enter the busy loop mode once new packet arrived. > > This interrupt mode will be helpful for both power management and > > resource utilization. Per my understanding, the application should > > turn off the task isolation before yield the CPU and the restart > > the task isolation after the new packet, right? > > Yes, exactly. > > I am not likely to pursue KVM myself, at least until the basic patch > series has been accepted upstream. Yes, and I will be more than happy to help on this. Thanks -jyh >